-- Logs begin at Thu 2022-06-16 18:27:24 UTC, end at Thu 2022-06-16 20:07:34 UTC. -- Jun 16 19:24:02.230633 dv-cinder-846173-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 19:24:02.237853 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 19:24:02 2022] *** Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: nodename: dv-cinder-846173-1-iscsi Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: machine: x86_64 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: clock source: unix Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: pcre jit disabled Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: detected number of CPU cores: 4 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: current working directory: / Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: detected binary path: /usr/bin/uwsgi-core Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: your processes number limit is 47680 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: your memory page size is 4096 bytes Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: detected max file descriptor number: 2048 Jun 16 19:24:02.241157 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: lock engine: pthread robust mutexes Jun 16 19:24:02.241930 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: thunder lock: enabled Jun 16 19:24:02.241930 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 19:24:02.241930 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 19:24:02.284905 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: Python main interpreter initialized at 0x560a6476a9c0 Jun 16 19:24:02.284905 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: python threads support enabled Jun 16 19:24:02.284905 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: your server socket listen backlog is limited to 100 connections Jun 16 19:24:02.284905 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: your mercy for graceful operations on workers is 90 seconds Jun 16 19:24:02.285283 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 19:24:02.285362 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: *** Operational MODE: preforking *** Jun 16 19:24:02.285601 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: *** uWSGI is running in multiple interpreter mode *** Jun 16 19:24:02.285601 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: spawned uWSGI master process (pid: 80294) Jun 16 19:24:02.285803 dv-cinder-846173-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 19:24:02.288302 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: spawned uWSGI worker 1 (pid: 80308, cores: 1) Jun 16 19:24:02.288302 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: spawned uWSGI worker 2 (pid: 80309, cores: 1) Jun 16 19:24:02.288302 dv-cinder-846173-1-iscsi devstack@keystone.service[80294]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 19:24:07.860153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 19:24:07.860153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: from cryptography.utils import int_from_bytes Jun 16 19:24:07.860951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 19:24:07.860951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: from cryptography.utils import int_from_bytes Jun 16 19:24:07.861371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 19:24:07.861371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: from cryptography.utils import int_from_bytes Jun 16 19:24:07.861482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 19:24:07.861482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: from cryptography.utils import int_from_bytes Jun 16 19:24:07.910491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 19:24:07.910679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 19:24:07.910779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 19:24:07.910865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] command line args: [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 19:24:07.910975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 19:24:07.911077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] command line args: [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 19:24:07.911147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 19:24:07.911318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ================================================================================ {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 19:24:07.911414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 19:24:07.911521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ================================================================================ {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 19:24:07.911605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] admin_token = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.911784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] admin_token = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.911873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] config_dir = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.911982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] config_dir = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] config_source = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] debug = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] config_source = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] debug = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.912998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.913943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_config_append = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_config_append = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_dir = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_dir = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_file = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_file = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.914950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.915959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.916954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.917064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.917142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:07.917303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.003235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.003535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.003807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.003899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] publish_errors = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.004922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] publish_errors = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.005870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] standard_threads = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] standard_threads = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.006906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] use_journal = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] use_json = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] use_stderr = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] use_journal = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] use_syslog = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] use_json = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] use_stderr = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.007941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] use_syslog = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.008178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 19:24:08.008446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.008956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.009968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.010989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.011988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.012864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.013870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.015850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.016993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.017946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.018991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.019979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.020933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.021899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.022843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.023849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.024980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.025969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.026942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.027947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.028992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.029974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.030955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.031922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.032886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.external = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.033850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.external = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.password = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.034844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] auth.token = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.password = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] auth.token = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.035990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.036949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.037938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.038945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.039994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.040996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.041973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.042978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.043062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.043222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.043302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.044997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.045960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.046970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.047972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.048985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.049946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.050900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.051894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.052875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.053979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.054976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.055929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.056966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.057937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.058906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.059662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.059754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.059913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.059996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.060964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.061958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.062963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.063926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.064892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.065852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.066816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.067840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] role.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.068971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] role.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] role.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] role.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.069948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.070966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.071944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.072947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.073903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.074911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.075907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.076886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.077846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.078821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.079839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.080972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.081941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.connection = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.connection = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.082833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.083989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.084944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.085993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 19:24:08.086625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.086859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 19:24:08.087054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80308) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 19:24:08.116315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.116436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.116526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.116603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.116706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.116793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.125642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.125753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.158771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.159022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.159109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.159298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.213086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.213482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.262884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.263138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.266515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.266758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.329765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.329997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.330990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.331109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.331326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.353659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.353894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.353978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.354959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.355044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.355185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.355402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.365976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.366375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.379971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80308) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.380830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80309) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 19:24:08.398051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.398265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 19:24:08.398551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.398634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.398766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 19:24:08.398864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 19:24:08.399071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 19:24:08.399399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.399903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.400090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.400346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.410667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.410961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.411231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.411349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.411509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.411633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.411906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.412226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.419821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.420591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.423119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.423398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.423679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.423938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.424021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.424824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.425897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.426080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.426350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.426618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.453716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.454672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.455359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.455639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.456336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.456622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.460927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.461322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.461606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.461936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.462330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.462615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.468960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.469242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.470019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.470305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.473754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.474040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.474322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.474607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.474815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.474943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.475957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.476234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.476506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.476776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.496249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.497069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.499786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.500084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.504725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.505545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.509276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.509686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.510102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.510511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.515421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.518652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.520468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.522403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.524180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.524481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.524764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.525045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.525322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.525601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.526077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.526377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.526659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.526959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.527238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.527514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.547480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.547768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.548041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.548321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.549266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.549550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.549818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.550081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.556319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.558006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.558148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.558303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.560036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.560314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.561814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.562098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.562380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.562675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.562984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.563519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.563811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.564093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.564382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.564666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.576499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.576778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.577072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.577358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.577733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.578004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.578142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.578279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.578405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.578558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.578838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.579239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.579518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.579793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.596933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.599313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.600252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.602599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.603623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.605359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.605988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.607624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.607914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.608196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.608353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.609895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.610173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.610453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.615565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.617250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.617752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.619409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.620485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.620781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.621174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.621457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.622661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.622971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.623361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.623640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.630906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.631221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.631507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.632843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.633130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.633410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.639509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80309) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.639789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.640070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.640342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.640615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80308) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 19:24:08.641558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.641972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.642883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.643037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.643162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80309) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.643318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.643597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.643868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.644138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.644408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.644681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.644949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80308) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 19:24:08.682084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560a6476a9c0 pid: 80309 (default app) Jun 16 19:24:08.683680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560a6476a9c0 pid: 80308 (default app) Jun 16 19:24:08.685805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799784f-de97-4c61-a787-8f04541ec2f8 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:08.686108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799784f-de97-4c61-a787-8f04541ec2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:08.686391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799784f-de97-4c61-a787-8f04541ec2f8 None None] PATH_INFO: `/v3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:08.690640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Thu Jun 16 19:24:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:17.412076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:17.412637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:17.412637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:17.417087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:17.422222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:17.435367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:17.435367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:17.492685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: INFO dbcounter [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] Registered counter for database keystone Jun 16 19:24:17.514555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG oslo_db.sqlalchemy.engines [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80309) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 19:24:17.520873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writer thread running {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 19:24:17.706632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG passlib.handlers.bcrypt [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80309) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 19:24:17.707047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG passlib.handlers.bcrypt [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80309) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 19:24:17.762405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:17.764083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:17.885856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 19:24:18.064914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:18.134463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: INFO dbcounter [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] Registered counter for database keystone Jun 16 19:24:18.153770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG oslo_db.sqlalchemy.engines [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80308) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 19:24:18.159875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writer thread running {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 19:24:18.276355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a501d-2300-4ef7-a13a-ec9fb739c1a4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:18.281130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:18.281788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:18.282078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:18.282358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:18.290303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:18.341906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 19:24:18.342194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800152e-848d-44f0-9ad2-2ff7eafc0214 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:18.351526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Thu Jun 16 19:24:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:24:18.357985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:18.372413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26677ccf-65bd-4ccd-9707-95d566f7d104 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:18.376834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:18.377937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:18.378234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:18.378516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] PATH_INFO: `/v3/projects/adf859e3c6024ed7b109d2f0ca77139a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:18.380068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] RBAC: Authorizing `identity:get_project(project_id=adf859e3c6024ed7b109d2f0ca77139a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:18.430085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 19:24:18.467484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980af45-35fe-4d32-be41-5f8b4e8f2cd6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:18.467484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Thu Jun 16 19:24:18 2022] GET /identity/v3/projects/adf859e3c6024ed7b109d2f0ca77139a => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:20.494827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416e29-dc1b-4858-ab83-fb5a3719521e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:20.495377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416e29-dc1b-4858-ab83-fb5a3719521e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:20.495471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416e29-dc1b-4858-ab83-fb5a3719521e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:20.499593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:20.504758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:20.505171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:20.505318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:20.523733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:20.525249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:20.533625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 19:24:20.556272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb416e29-dc1b-4858-ab83-fb5a3719521e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:20.568299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb416e29-dc1b-4858-ab83-fb5a3719521e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:20.572478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:20.573101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:20.573388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:20.573666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:20.581206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:20.582991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d06c-da65-4bab-b507-759f11710fef admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:20.592000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Thu Jun 16 19:24:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:24:20.596878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:20.605621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8725e-2624-48c7-acf4-6ac903bfce8f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:20.610484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:20.611368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:20.611652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:20.611930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:20.613319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:20.615170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768931b-6ed5-4a0e-99f7-aeca01ac1fc8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:20.616719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Thu Jun 16 19:24:20 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:22.765373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ebf35-ddf4-42fa-893e-24f6ce90c980 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:22.765918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ebf35-ddf4-42fa-893e-24f6ce90c980 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:22.765918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ebf35-ddf4-42fa-893e-24f6ce90c980 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:22.767368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:22.773332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:22.773903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:22.773996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:22.792811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:22.794178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:22.836077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d8cb-8335-4c7c-853c-afb83d60c2cf None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:22.836370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d8cb-8335-4c7c-853c-afb83d60c2cf None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:22.836652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d8cb-8335-4c7c-853c-afb83d60c2cf None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:22.838000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:22.843553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:22.843848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:22.844131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:22.885488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:22.889094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG passlib.handlers.bcrypt [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80308) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 19:24:22.889517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG passlib.handlers.bcrypt [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80308) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 19:24:22.909501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:22.933264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:22.935870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:22.937467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:23.014062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741e7cc-cdc7-4079-834d-15101f71f016 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.035563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:23.047702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.048505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.048795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.049078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.057608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.061054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:23.072209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:23.073124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:23.074009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:24:23.079099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.156993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6dc71-a62b-4fa2-93ea-d71b88bbda9d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.158657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.159268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.180669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f686f-80cc-4fd3-8155-694cf0a7b20c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.193921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.199524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.199524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.199524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.215097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.215759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.216069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.216364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.217143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:23.217455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.218850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.230385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:23.236420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.261813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.263393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.333102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7f88-0367-4228-8150-f6e1e516d1ed None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.361015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:24:23.366518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.368540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.369546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.369656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.369754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.369960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.371200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.372687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.374291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:23.379408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.393585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.464859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a4a89-69c1-48d9-942d-55e735940110 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.476571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583dfe0-a8ed-4674-89c2-fd73642c8b04 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.498179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.499003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.499292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.499575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.509671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.510462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.510749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.511141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.512334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.513787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.515653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:23.517875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.519359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.521081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:24:23.540585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:23.541556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:23.542395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:24:23.542395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:24:23.542395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:24:23.542395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:23.542395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:24:23.547852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:23.599823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.602834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:23.619102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4b7b1-8dcb-42b6-ac16-3d5d07041eca None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.647347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f2fa-76d9-4a36-95c7-2145c8c48dbe None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:23.654738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.657346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.657346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.657346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.657346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.domain.id=default, user.id=07dde671a0d34c9484dac362c8255554) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:23.657346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.658529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1300d-5d11-4b96-86a3-9f679d510fed None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.665509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/role_assignments?user.id=07dde671a0d34c9484dac362c8255554&scope.domain.id=default&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:23.681952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:23.682649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:23.682970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:23.683255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:23.684006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:23.684320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:23.686072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f876c7b-bc49-4e52-a561-3dc4df1de113 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:23.706347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:26.005074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e6cd3-fd82-45ec-8e82-df462035bde6 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.005589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e6cd3-fd82-45ec-8e82-df462035bde6 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.005662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e6cd3-fd82-45ec-8e82-df462035bde6 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.007187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:26.012507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.012937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.013105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.033144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:26.034354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:26.131270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 11/23] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:26.166601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e6cd3-fd82-45ec-8e82-df462035bde6 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:26.325403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e6cd3-fd82-45ec-8e82-df462035bde6 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:26.369473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:26.370271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.370562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.370845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.386833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:26.394605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:26.395497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:26.396257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 13/24] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:26.403256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:26.491457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:26.572164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1250-62cd-450d-ab0f-f6960bd0fb85 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:26.626368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:26.626994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.627279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.627558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.628318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:26.628627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:26.630003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98e37-27b7-41da-92a6-f0fe71e3245c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:26.650506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 12/25] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:26.660004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:26.662392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606a27-9a5b-4ad7-97a3-3f987e4d16d8 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.662687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606a27-9a5b-4ad7-97a3-3f987e4d16d8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.663052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606a27-9a5b-4ad7-97a3-3f987e4d16d8 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.664872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:26.666493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:26.687280 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8a578-0407-4998-a3c6-bc99c8b4c2ae None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.687895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8a578-0407-4998-a3c6-bc99c8b4c2ae None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.688334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8a578-0407-4998-a3c6-bc99c8b4c2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.723963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f0e8a578-0407-4998-a3c6-bc99c8b4c2ae None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:26.725210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f0e8a578-0407-4998-a3c6-bc99c8b4c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:26.801290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ef03-6844-4933-8d25-512133ecdee0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:26.826049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:26.827805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97637d0-66a4-4879-9b22-8b8f1e531d6c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.828247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97637d0-66a4-4879-9b22-8b8f1e531d6c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.828698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97637d0-66a4-4879-9b22-8b8f1e531d6c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.830245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 15/28] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:26.831513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbf234-39c9-4f30-af62-b6c82fcba775 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.831968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbf234-39c9-4f30-af62-b6c82fcba775 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.832389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbf234-39c9-4f30-af62-b6c82fcba775 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.833751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 16/29] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:26.838832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.839340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.839768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.852142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:26.854303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.854596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.854882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.856183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:26.857701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5586455-fc77-48cc-a18f-d8a52972907e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:26.859366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 14/30] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:26 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:26.864756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:26.865052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:26.865336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:26.867012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:26.868386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:26.882895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:26.884257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:26.907046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:26.957576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 17/31] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:26.963821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:26.977858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 15/32] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:26.978966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:26.985731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:27.091223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bf6a-4860-4a31-a80c-4fa247973ae4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.110417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9247-6431-48be-ac98-156811c2c4d4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.128017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.128985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.129571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.130061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.131140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.132708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:27.149200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.149929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.150217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.150498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.151695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.153166 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:27.155000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 16/33] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:27.159446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:27.237900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:27.275626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8cc9b-2059-4b8f-89c8-0532710f434f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.312126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.313254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.313785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.314265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.318418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:27.370135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 18/34] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Jun 16 19:24:26 2022] POST /identity/v3/roles => generated 225 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:27.370135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:27.438131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:27.619245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee028e-9cd2-41a2-bc13-66c646145281 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.624487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 17/35] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Jun 16 19:24:27 2022] POST /identity/v3/roles => generated 219 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:27.625292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:27.785367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.785367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.785367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.785367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.792575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a31ab-1532-46e4-90db-dd1e310d7788 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.798368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.800166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6d01ea25-de1b-4c70-b899-785af8dd7367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:24:27.824750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:27.825692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:27.826494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:24:27.826494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:24:27.826494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:24:27.826494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:27.826494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 19/36] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:27.831437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067ab6-6118-47ed-b44d-6df52f4bc6f1 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.831437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067ab6-6118-47ed-b44d-6df52f4bc6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.831437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067ab6-6118-47ed-b44d-6df52f4bc6f1 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.833786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 20/37] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:27.835701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877115b4-c268-4551-8554-80371ec948b7 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.835996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877115b4-c268-4551-8554-80371ec948b7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.836276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877115b4-c268-4551-8554-80371ec948b7 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.837818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 21/38] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:27.838380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877115b4-c268-4551-8554-80371ec948b7 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:27.846007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:27.851631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.852536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.852835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.853288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] PATH_INFO: `/v3/domains/default/users/07dde671a0d34c9484dac362c8255554/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.855638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=07dde671a0d34c9484dac362c8255554, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.857948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127bfa5-90ba-4e86-bb37-e7e4b3f2b3ba None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:27.928207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:27.947880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877115b4-c268-4551-8554-80371ec948b7 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:27.959763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 18/39] 10.222.0.36 () {58 vars in 1361 bytes} [Thu Jun 16 19:24:27 2022] PUT /identity/v3/domains/default/users/07dde671a0d34c9484dac362c8255554/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:27.961209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.961512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.961801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.980658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:27.982983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:27.983956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:27.984653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:27.984947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:27.985283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:27.986051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:27.986370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:27.987872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b80cb-96f7-41e4-a8f9-5cba78019b0b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:28.000763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 22/40] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:28.002194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:28.002485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:28.002772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:28.020407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:28.022039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:28.066572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 19/41] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:28.067390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:28.102664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 23/42] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:28.103651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:28.185610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:28.200383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e31ce-10c4-4c45-9368-10cdf750439f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:28.216750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd647d-d0ba-4890-810a-f8a9b1caa1b1 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:28.226602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:28.241314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:28.241989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:28.242278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:28.242562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:28.243321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:28.244853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:28.253485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:28.254573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:28.255073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:28.255566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:28.257190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:28.258822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:28.260830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 24/43] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:28.261776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:28.361998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 20/44] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:24:28 2022] POST /identity/v3/projects => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:28.363099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:28.373209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26eb1a-d58a-48e7-ac5c-7eba13e809b1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:28.424000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:28.425294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:28.425780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:28.426253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:28.427260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:28.429034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d15b39-4b0d-4b4c-a41e-fc9c7f03308a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:28.490334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3165-6618-4185-93d7-78d78e72344d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:28.529546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:28.530273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:28.530572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:28.530860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:28.531645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:28.533121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f865d7d-6440-4c6e-b1ef-380a2368b4a6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:28.618298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:28.624783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 25/45] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Jun 16 19:24:28 2022] POST /identity/v3/roles => generated 223 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:28.635966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 21/46] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:24:28 2022] POST /identity/v3/projects => generated 312 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:30.111816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038bbeb-d562-457c-bfd6-4e3f8e470a61 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.112332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038bbeb-d562-457c-bfd6-4e3f8e470a61 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.112413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038bbeb-d562-457c-bfd6-4e3f8e470a61 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.113764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 26/47] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:30.118943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.119375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.119521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.136197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:30.137385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:30.154588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:30.203615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 22/48] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:30.226258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038bbeb-d562-457c-bfd6-4e3f8e470a61 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.327744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038bbeb-d562-457c-bfd6-4e3f8e470a61 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:30.360168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:30.360968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.361261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.361546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.373726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:30.382418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:30.383299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:30.384064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 27/49] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:30.389967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.487654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9a8ee-8a35-4c7b-821a-6f5708b5003b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:30.521063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:30.521736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.522026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.522308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.523104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:30.523419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:30.524825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:30.532931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 23/50] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:30.538427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.595189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:30.636361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a01f2-5c84-4934-a41b-215b083f1bc3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:30.668689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:30.670099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.670390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.670670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.672152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:30.673626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cac2-db6a-40bd-ae8f-537679de78e0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:30.675294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 28/51] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:30.680594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.699565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:30.783432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d7be-6de7-4a97-899c-3e450ea9ba48 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:30.799938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd23e6-1125-4350-a09c-e7f1e85e933a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.800311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd23e6-1125-4350-a09c-e7f1e85e933a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.800666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd23e6-1125-4350-a09c-e7f1e85e933a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.802045 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 29/52] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:30.806744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.807074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.807358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.817912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:30.818706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.819038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.819326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.820500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:30.821951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:30.823597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 24/53] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:30.824009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:30.825210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:30.829244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.892390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 30/54] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:30.910480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:30.915556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:30.927994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca48287-a539-4616-8f96-0b24638a9bc8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:30.971196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:30.973450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:30.974116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:30.974408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:30.974685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:30.975547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.domain.id=default, user.id=07dde671a0d34c9484dac362c8255554) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:30.975867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:30.977366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:30.983938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 25/55] 10.222.0.36 () {56 vars in 1322 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/role_assignments?user.id=07dde671a0d34c9484dac362c8255554&scope.domain.id=default&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:31.016393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a9de-52c8-4609-8342-98ce97c24df1 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:31.052883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:31.053859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:31.054147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:31.054427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:31.055685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:31.057163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c604e-54b4-45be-bf89-2375a0ca39d8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:31.058832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 31/56] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:31.065003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:31.167228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af26610-7db9-4f2b-8a7b-749ed27b0482 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:31.204914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:31.205573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:31.205861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:31.206139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:31.206872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:31.208341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2afc2-c8b8-4fc5-b816-4bde976885fd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:31.298616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:31.304516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 26/57] 10.222.0.36 () {60 vars in 1140 bytes} [Thu Jun 16 19:24:31 2022] POST /identity/v3/projects => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:33.438286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a4e7-89ca-4ca0-939c-c541d2cfcad8 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:33.438782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a4e7-89ca-4ca0-939c-c541d2cfcad8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:33.438782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a4e7-89ca-4ca0-939c-c541d2cfcad8 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:33.440168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 32/58] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:33.445317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:33.445725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:33.445888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:33.464250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:33.465511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:33.529865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 27/59] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:33.554487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a4e7-89ca-4ca0-939c-c541d2cfcad8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:33.652955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a4e7-89ca-4ca0-939c-c541d2cfcad8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:33.675778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:33.686821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:33.687645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:33.687931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:33.688211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:33.689427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:33.690835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6e64b-3c33-4cad-9d9b-d0cd14963a2a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:33.692450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 33/60] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:33.697811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:33.791572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c6a50-da7d-4b62-bd6d-97799842d97e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:33.819790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:33.825281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:33.825965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:33.826252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:33.826530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:33.827272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:33.828774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caf0bf-d360-4fac-b2a3-ce9f1ee02305 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:33.937018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 28/61] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:33 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:37.040805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf0fa-aaf3-4a3a-bdbd-91cc61f48905 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.041319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf0fa-aaf3-4a3a-bdbd-91cc61f48905 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.041393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf0fa-aaf3-4a3a-bdbd-91cc61f48905 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.043059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 34/62] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:37.048461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.048864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.049040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.071933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:37.073157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:37.144156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 29/63] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:37.178248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bf0fa-aaf3-4a3a-bdbd-91cc61f48905 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:37.324684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bf0fa-aaf3-4a3a-bdbd-91cc61f48905 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:37.373129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:37.373959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.374246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.374527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.389171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-7237f7f6-96f7-46f0-99a1-643e8ff80d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:37.396864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:37.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:37.398523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 35/64] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:37.404019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:37.532964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5b43f-fa48-4ace-93ee-515280a2a289 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:37.562226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:37.575073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7963c-bf97-441b-9aec-6502f09599d5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.575343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7963c-bf97-441b-9aec-6502f09599d5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.575602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7963c-bf97-441b-9aec-6502f09599d5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.577040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 36/65] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:37.579223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6cccd-b606-4d2c-b898-0f7cf9ac6e74 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.579520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6cccd-b606-4d2c-b898-0f7cf9ac6e74 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.580075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6cccd-b606-4d2c-b898-0f7cf9ac6e74 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.585771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 37/66] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:37.585771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.585771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.585771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.589016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:37.589955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.590397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.590859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.592394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:37.592718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:37.594187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6313-6208-4864-8ac7-1cd1aaed265e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:37.603821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 30/67] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:37.606769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c985d5-6d7e-4219-9037-bf60285eb71a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.607703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c985d5-6d7e-4219-9037-bf60285eb71a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.607999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c985d5-6d7e-4219-9037-bf60285eb71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.608655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:37.610217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:37.627320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-61c985d5-6d7e-4219-9037-bf60285eb71a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:37.629021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-61c985d5-6d7e-4219-9037-bf60285eb71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:37.665012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:37.687375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 38/68] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:37.688096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:37.716042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 31/69] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:37.724291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543148a-175f-43d5-afb7-7a28a64b75b4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.724590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543148a-175f-43d5-afb7-7a28a64b75b4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.724874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543148a-175f-43d5-afb7-7a28a64b75b4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.726400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 32/70] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:37.726994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543148a-175f-43d5-afb7-7a28a64b75b4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:37.804116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062e3f0-0863-4ba1-8e12-8721f03f7d29 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:37.839298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:37.840095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.840386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.840666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.842392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:37.843874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441430-d700-41dd-a862-b6fb30a2ab96 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:37.845501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 39/71] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:37.846642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e370-a6d8-4cc3-ae0d-dbd7e588c33e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.846955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e370-a6d8-4cc3-ae0d-dbd7e588c33e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.847236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e370-a6d8-4cc3-ae0d-dbd7e588c33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.850487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543148a-175f-43d5-afb7-7a28a64b75b4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:37.860590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:37.864735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5bc8e370-a6d8-4cc3-ae0d-dbd7e588c33e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:37.865935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5bc8e370-a6d8-4cc3-ae0d-dbd7e588c33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:37.886780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:37.887621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.887909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.888227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.900338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:37.909633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:37.910663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:37.911515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 33/72] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:37.912130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:37.914020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:37.936019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 40/73] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:37.937651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0431b-1613-43e4-9348-0bbbe396928c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:37.937948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0431b-1613-43e4-9348-0bbbe396928c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:37.938230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0431b-1613-43e4-9348-0bbbe396928c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:37.939665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 41/74] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:37.940286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0431b-1613-43e4-9348-0bbbe396928c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.025741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263865e-5e3e-49c3-82e0-47d8b5a0ca51 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.040421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0431b-1613-43e4-9348-0bbbe396928c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.058809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.059639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.060334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.060620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.070852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:38.073534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.074186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.074479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.074763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:38.078958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:38.079841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:38.080642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 34/75] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:38.084271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.112982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.123382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.143404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.144879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.191269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba7713-b20c-49e0-81c8-2fad78b9d428 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.225264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.225926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.226213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.226496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.227286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:38.227597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.228987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fe386-6437-46d0-9a2a-7e6a3511af56 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.237196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 35/76] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.238385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.238681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.238996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.245233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 42/77] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:38.246091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.254822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:38.256039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:38.320345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 36/78] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:38.320973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.331204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.347708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.351378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdb108-ce62-4b92-98ee-46efa4047971 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.386003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.386773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.387089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.387371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.397837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:38.405561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:38.406539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:38.407561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 43/79] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:38.408348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.427526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37f1b5-d443-48be-8761-da5ea3f5730e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.460984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.461651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.461941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.462221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.462997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:38.463315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.464709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.472534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 37/80] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.473310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.508500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb72631-cc4e-4e3d-ba3f-e26d186f1dbc None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.542017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.542888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.543201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.543478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.551026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.554008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.559001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.561027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.562960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 44/81] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.563734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.569490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3353c-4c25-42df-9b8c-abf25c9dc347 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.605888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.606746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.607217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.607510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.608715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.610202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.611929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 38/82] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.612687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.670973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe0d8c-e6cf-466d-9260-38e3292b2414 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.709241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.710047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.710338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.710618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.712103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.714329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.717576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 45/83] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.719163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.720225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee971e1-e704-4df1-8fc9-c76d86fbfbfb None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.756417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.757084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.757373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.757648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.758404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:38.758713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.761212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.769687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 39/84] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.770604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.786603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.797635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:38.834053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740290-d99a-4613-8d82-44b3eae9c52b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.871691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.873127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.873444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.874569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.874997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.875301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e933f5-bd8d-4f03-99d4-f41e7fbbb529 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.876475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:38.878125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 46/85] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:38.878908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:38.913427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:38.914201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:38.914488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:38.914768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:38.987133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef81a0b-475e-4327-aba8-791800cf7a17 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:38.996486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:38.998558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.013065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.024996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.025720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.026004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.026282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.027165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=07dde671a0d34c9484dac362c8255554) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:39.027483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.029467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.031976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.040022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 47/86] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/role_assignments?user.id=07dde671a0d34c9484dac362c8255554&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:39.041965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.125508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 40/87] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:39.127163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.183077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cd3bc-f917-429e-aed4-faa6ae317747 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.229651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.230352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.230641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.230942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.231683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.233176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.266378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3d25f-e387-44fe-ad3c-ac01b9a4892c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.279979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.302686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.303612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.303902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.304185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.305404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.306861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.308674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 41/88] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:39.309386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.323273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 48/89] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:39 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:39.323273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.397439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.451886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc4157-cea7-43d3-825d-0f01a458f119 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.453254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface543-e8f9-4461-afc9-1fc300dfb398 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.495335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.496384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.496667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.496994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.498309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.499831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.507075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 49/90] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:39.507075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.549349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.551289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.551816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.552298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.614470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.630559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29021-6f49-434e-a300-26e06afb4a5c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.640098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.642258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.661515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.680504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.680718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.680941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.681167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.770820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.772737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:39.777645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 42/91] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:39.778334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.875704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.895672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a5952-fa65-4d5c-8a85-89490d3bd2cf None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:39.912321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 50/92] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:39.912321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:39.936112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:39.988293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:39.989213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:39.989646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:39.990071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:39.991116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:39.991599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:39.993232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:40.009974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 43/93] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:40.011094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:40.021248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b4e67-d61d-4a6d-ae79-06c36982d803 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:40.064867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:40.065749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:40.066033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:40.066311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:40.067558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:40.069601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:40.071717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 51/94] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:39 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:40.078801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:40.194516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:40.213473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb58f1-0ce7-48b3-b225-57879442f881 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:40.264869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90bcfa-bbbf-4d9a-a9f0-5894b98fc137 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:40.304174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:40.305122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:40.305552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:40.305974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:40.307000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:40.307479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:40.309269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:40.320457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 52/95] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:40 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:40.322595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:40.323679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:40.324140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:40.324567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] PATH_INFO: `/v3/projects/e5be27e98a4b4cf78bda910d7f100fea` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:40.326111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] RBAC: Authorizing `identity:get_project(project_id=e5be27e98a4b4cf78bda910d7f100fea)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:40.327867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a6f5e-4a94-426b-b78e-0f12daaa23ce None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:40.329897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 44/96] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:40 2022] GET /identity/v3/projects/e5be27e98a4b4cf78bda910d7f100fea => generated 312 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:40.336208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:40.481804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe611-17ab-4f97-b5c7-714879ddc7aa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:40.531676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:40.532594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:40.533061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:40.533496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:40.534496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=e5be27e98a4b4cf78bda910d7f100fea, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:40.534982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:40.536648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b5f6d-b83e-42d9-89da-5cb58c4622e1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:40.546117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 53/97] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:40 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=e5be27e98a4b4cf78bda910d7f100fea&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:41.864928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c763f-8887-4879-9b73-523fb7bf6c66 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:41.865786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c763f-8887-4879-9b73-523fb7bf6c66 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:41.866218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c763f-8887-4879-9b73-523fb7bf6c66 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:41.867793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 45/98] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:41.877099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:41.877861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:41.878295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:41.899613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:41.901056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:41.918820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:41.984349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 54/99] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:42.023092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c763f-8887-4879-9b73-523fb7bf6c66 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:42.040936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:42.151577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c763f-8887-4879-9b73-523fb7bf6c66 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:42.212771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:42.213584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.213874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.214153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.228399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-41039ed1-bf16-41f4-be79-266f14979b60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:42.236451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:42.237379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:42.238147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 46/100] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:42.245572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:42.345367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908cb4-1592-4db2-8d6a-f490e697fea8 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.345686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908cb4-1592-4db2-8d6a-f490e697fea8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.345961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908cb4-1592-4db2-8d6a-f490e697fea8 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.347337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 47/101] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:42.352529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce220d5-7658-47d0-a37a-792583a7a687 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.352825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce220d5-7658-47d0-a37a-792583a7a687 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.353112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce220d5-7658-47d0-a37a-792583a7a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.380421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4ce220d5-7658-47d0-a37a-792583a7a687 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:42.381633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4ce220d5-7658-47d0-a37a-792583a7a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:42.400221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3181a8-477e-491a-bb2e-35039d9d6448 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:42.460519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:42.461179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.461472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.461754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.462531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:42.462845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:42.464273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b78636-ba38-4f99-8e35-17bdc00bf76c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:42.492497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 55/102] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:42.496588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f13e-e554-4b74-8894-e40462775d6b None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.496892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f13e-e554-4b74-8894-e40462775d6b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.497183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189f13e-e554-4b74-8894-e40462775d6b None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.500407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 56/103] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:42.501155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189f13e-e554-4b74-8894-e40462775d6b None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:42.524987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 48/104] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:42.526322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.526622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.526908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.533332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:42.540201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:42.565647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:42.567411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:42.689121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189f13e-e554-4b74-8894-e40462775d6b None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:42.713085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 49/105] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:42.713713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:42.743126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:42.744453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.744762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.745053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.748250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:42.749840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:42.751563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 57/106] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:42.755032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:42.855308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:42.868254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001b8d-7335-4220-b1e2-304ca6ae77f5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:42.931053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:42.932649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:42.933076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:42.933494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:42.945202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:42.949054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e795e-ad2f-4b42-84e2-a52cf04ad121 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:42.954054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:42.955151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:42.956094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:42.958940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 58/107] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:24:42.959901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.037375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.050044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.051082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.051568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.052006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.053443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.055077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.057352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 50/108] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:43.058285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.128474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3088-694b-4301-9f96-5c8560782dc1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.169988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf862-4545-46cc-9e31-4727de03df69 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.191831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.192864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.193009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.193303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.199759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.201729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.203512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 59/109] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:42 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:43.204710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.208265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.209197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.209679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.210151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.211263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:43.211825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.213386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.222372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.224012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 51/110] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:43.225022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.288856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.333295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6118c02-ee54-4ea7-b588-37218f574d5e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.368789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.370036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.370518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.371068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/07dde671a0d34c9484dac362c8255554/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.373362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, user_id=07dde671a0d34c9484dac362c8255554, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.375121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.399061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bd21c-ab69-4582-b401-7115d18f800e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.438098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.438773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.439253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.439689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.440541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.443018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4defa-c767-4f33-b93a-125991397925 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.469527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 52/111] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:43 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/07dde671a0d34c9484dac362c8255554/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:43.470721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.577968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 60/112] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:24:43 2022] POST /identity/v3/projects => generated 302 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:43.579931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef25-3156-42d5-8c23-63b4ecab7fee None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.580526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef25-3156-42d5-8c23-63b4ecab7fee None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.582006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef25-3156-42d5-8c23-63b4ecab7fee None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.584437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 61/113] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:43.586953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d795-3123-4fd2-9fc5-db40803fe280 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.587566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d795-3123-4fd2-9fc5-db40803fe280 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.588054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d795-3123-4fd2-9fc5-db40803fe280 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.590636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 62/114] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:43.592552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.593100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.593673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.618190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:43.619945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:43.635117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce4b4-df70-482d-ac3b-31862da95016 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.670676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.684265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.685871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.686160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.692705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] PATH_INFO: `/v3/users/7249eadb739a42e5b4d211e9be7b5424` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.696367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.704691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] RBAC: Authorizing `identity:get_user(user_id=7249eadb739a42e5b4d211e9be7b5424)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:43.706187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac434c-b1ed-483e-a619-70b74e3cc596 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:43.708005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 53/115] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:43 2022] GET /identity/v3/users/7249eadb739a42e5b4d211e9be7b5424 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:43.711297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4687d2-1231-450b-b48b-a7c1f349bc09 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.711297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4687d2-1231-450b-b48b-a7c1f349bc09 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.711297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4687d2-1231-450b-b48b-a7c1f349bc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.723237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 63/116] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:43.724468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.736010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fe4687d2-1231-450b-b48b-a7c1f349bc09 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:43.737431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fe4687d2-1231-450b-b48b-a7c1f349bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:43.838506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 54/117] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:43.839677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb945-f177-4dfc-9da6-98769c476c99 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.839977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb945-f177-4dfc-9da6-98769c476c99 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.840265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb945-f177-4dfc-9da6-98769c476c99 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.841714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 55/118] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:43.842276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb945-f177-4dfc-9da6-98769c476c99 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:43.859301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574bfb-4e55-41b5-a288-1ce019f20c87 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.915420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:43.916242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:43.916567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:43.917038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:43.969658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb945-f177-4dfc-9da6-98769c476c99 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:43.982708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:43.993652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.010014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.011222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.011642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.012231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.012636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.012922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.026053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-6b9a7253-3631-4d2b-ac2d-0b4d17439e6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:44.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:44.035551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:44.037209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 56/119] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:44.038589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.039708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.039708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.066004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:44.068243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:44.135830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 64/120] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:44.136964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.166945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 57/121] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:44.166945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.261098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.274379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9194b62-34e2-4750-b4cd-02ccc9e33b79 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.292029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.296074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81312e6-6946-4e29-a360-390c22a6bfb5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.312340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.313348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.313741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.314128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.325591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:44.335140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:44.336164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:44.336949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:44.337962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 65/122] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:44.338446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.338563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.339548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.339839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.340121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.340893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:44.341207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.343182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.353238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 58/123] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.354082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.478272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123cec6-1651-4a5a-b3a9-d5019656a5ec None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.483419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fcbd7-0b0d-4169-849e-5f9fb2cf8007 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.527382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.528183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.528445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.528587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.528747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.530480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.530988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.531647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.533375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.535535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.537937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 66/124] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.539132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.540391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:44.550241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:44.551184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:44.552001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 59/125] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:44.552664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.565294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.584860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.654285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b234c-b97c-4c45-957d-e496eafd404f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.666843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c2197-6f9d-44d3-9b7b-5cdf2db2d6e6 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.692567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.693343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.693732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.694250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.695352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:44.695665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.697137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.706612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 67/126] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.707414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.707970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.709030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.709457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.709877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.711648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.713291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.715264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 60/127] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.716997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.791690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.813607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:44.815690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefee4bb-edab-420a-b14d-3dbcf2e1a42f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.823698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e902889-6094-4c35-882e-3a894b4c3af7 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:44.852187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31543604-d119-474d-b373-3386b282d919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.852986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31543604-d119-474d-b373-3386b282d919 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.853386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31543604-d119-474d-b373-3386b282d919 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.853698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31543604-d119-474d-b373-3386b282d919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.854743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31543604-d119-474d-b373-3386b282d919 None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=7249eadb739a42e5b4d211e9be7b5424) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:44.855166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31543604-d119-474d-b373-3386b282d919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.856914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31543604-d119-474d-b373-3386b282d919 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.864704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 68/128] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/role_assignments?user.id=7249eadb739a42e5b4d211e9be7b5424&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.865631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31543604-d119-474d-b373-3386b282d919 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.866592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:44.867541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:44.868068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:44.868640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:44.869724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:44.870245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:44.871908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:44.882437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 61/129] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:44.884004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:44.994460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31543604-d119-474d-b373-3386b282d919 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.007402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db082ee-f72d-43fe-b903-471c02e72c0e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.032691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.033705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.034062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.034410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.035899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.037564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.039842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 69/130] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:45.040904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.049719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.051048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.051514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.051973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] PATH_INFO: `/v3/projects/e5be27e98a4b4cf78bda910d7f100fea` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.053429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] RBAC: Authorizing `identity:get_project(project_id=e5be27e98a4b4cf78bda910d7f100fea)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.054405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:45.056469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.058672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 62/131] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:44 2022] GET /identity/v3/projects/e5be27e98a4b4cf78bda910d7f100fea => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:45.059866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.085185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:45.154947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ae101-c9a3-4e39-bcf9-5263bbca59f0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.181409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19eeae-1e5f-4fc7-ad21-5333dc69e424 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.200742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.202434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.202905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.203351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.204786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.206369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.208583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 70/132] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:45 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:45.209474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.222194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.223312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.223698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.224077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.225385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.227298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.229100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 63/133] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:45 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:45.230112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.293863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:45.317015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d32-c745-45d6-aefb-3064d61e948a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.346497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:45.358699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db36d6-5dcb-4f38-935f-ce0c9166b4c9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.389024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.390757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.391303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.391778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] PATH_INFO: `/v3/projects/e5be27e98a4b4cf78bda910d7f100fea/users/dffa90611fbe4b429e2703e120177f8e/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.394050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5be27e98a4b4cf78bda910d7f100fea, user_id=dffa90611fbe4b429e2703e120177f8e, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.395742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.396031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.396904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.397195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.397482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.398669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.400305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.407635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 64/134] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:45 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:45.407635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.505460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 71/135] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:45 2022] PUT /identity/v3/projects/e5be27e98a4b4cf78bda910d7f100fea/users/dffa90611fbe4b429e2703e120177f8e/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:45.506616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:45.514323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199fbf7-8d25-4ee8-8f72-c1a5e1c5d848 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.553696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.554610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.554946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.555693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/dffa90611fbe4b429e2703e120177f8e/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.557719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, user_id=dffa90611fbe4b429e2703e120177f8e, role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.566958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8343371-c23e-4930-a946-d3e2c8ff3710 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.634567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 65/136] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:45 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/dffa90611fbe4b429e2703e120177f8e/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:45.718273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8f78-b143-461c-b615-d309153c2ca5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:45.760118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:45.761048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.761342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.761628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/dffa90611fbe4b429e2703e120177f8e/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.763952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, user_id=dffa90611fbe4b429e2703e120177f8e, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:45.765448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829cde-e180-4ddc-aeba-0bba2c0a1e90 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:45.869974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:45.875532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 72/137] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:45 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/dffa90611fbe4b429e2703e120177f8e/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:45.991321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4d780-de97-4629-acf4-75e9fa09e4c1 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.991643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4d780-de97-4629-acf4-75e9fa09e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.991901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4d780-de97-4629-acf4-75e9fa09e4c1 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:45.993307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 66/138] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:45.999185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb0e3-b608-452d-99dc-e88c354a819f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:45.999712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb0e3-b608-452d-99dc-e88c354a819f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:45.999788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb0e3-b608-452d-99dc-e88c354a819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.024182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3c6cb0e3-b608-452d-99dc-e88c354a819f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:46.025547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3c6cb0e3-b608-452d-99dc-e88c354a819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:46.112303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 73/139] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:46.146786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4d780-de97-4629-acf4-75e9fa09e4c1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.177400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:46.246306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4107112-cd7f-417f-b142-90ae7c11f144 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.246306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4107112-cd7f-417f-b142-90ae7c11f144 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.246306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4107112-cd7f-417f-b142-90ae7c11f144 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.246306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 74/140] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:46.256008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.256308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.256602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.285992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:46.287211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:46.292462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4d780-de97-4629-acf4-75e9fa09e4c1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.343373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.344241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.344534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.344821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.368342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:46.378006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:46.378862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:46.379660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:46.383796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 67/141] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:46.390229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.408724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 75/142] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:46.437004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.499818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:46.525335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:46.527163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc0279-ec3c-4209-b378-2f7de7461f55 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.568067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.569124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.569608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.570084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.571208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:46.571751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:46.573463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:46.584188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 68/143] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:46.588029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da28ed-eb98-48bf-bf4e-59c36097c8a7 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.590200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.645027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.645845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.646137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.646475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.651709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:46.653200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:46.655100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 76/144] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:46.664081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.709880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dbfe4-f06e-461d-b5ae-efb621e80d04 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.750472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.751777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.752259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.752742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] PATH_INFO: `/v3/users/07dde671a0d34c9484dac362c8255554` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.755842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] RBAC: Authorizing `identity:get_user(user_id=07dde671a0d34c9484dac362c8255554)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:46.758402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:46.760533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 69/145] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:46 2022] GET /identity/v3/users/07dde671a0d34c9484dac362c8255554 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:46.767379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:46.788706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:46.872137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980078d-3109-4a40-8b76-ecbe8542b1cf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.912448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c255d-14c6-4de4-ae9c-5fa9a558154d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:46.917543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:46.931514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.932219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.932480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.932761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:46.933486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:46.934963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:46.956618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:46.957676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:46.958203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:46.958694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:47.034555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:47.036346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:47.044794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:47.067025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 77/146] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:46 2022] POST /identity/v3/users => generated 283 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:24:47.150493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 70/147] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:47.160065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:47.323682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209f0c0-49aa-4541-a690-ad84912c06af None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:47.358551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:47.359717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:47.360169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:47.360589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:47.361973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:47.363605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0129-3891-4f24-b6fa-650e174bab4d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:47.365812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 78/148] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:47 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:47.372342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:47.483290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984f8b9-bf1c-4480-80f8-3817187db1fc None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:47.490971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:47.524627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:47.525292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:47.525577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:47.525925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:47.526893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=07dde671a0d34c9484dac362c8255554) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:47.527236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:47.528695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17e038-4e31-48c6-83d5-04e37aac42cc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:47.535024 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 71/149] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:47 2022] GET /identity/v3/role_assignments?user.id=07dde671a0d34c9484dac362c8255554&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:47.824848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452bac4-a462-4dbf-8dac-bfdb9ee0857a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:47.825532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452bac4-a462-4dbf-8dac-bfdb9ee0857a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:47.826063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452bac4-a462-4dbf-8dac-bfdb9ee0857a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:47.828050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 79/150] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:47.833058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:47.833351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:47.833632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:47.855013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:47.856218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:47.939288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 72/151] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:47.962238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452bac4-a462-4dbf-8dac-bfdb9ee0857a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:48.027111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:48.085578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452bac4-a462-4dbf-8dac-bfdb9ee0857a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:48.121840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:48.122724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:48.123071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:48.123353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:48.135307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:48.143690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:48.144594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:48.145379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 80/152] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:48.152556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:48.272130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:48.277836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526020d-6e2a-49cc-9e31-72327ad4ac3e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:48.317393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:48.318076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:48.318360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:48.318636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:48.319432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:48.319761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:48.321185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:48.332473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 73/153] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:48.338236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:48.457818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174fa81-acb9-4cf4-aab1-e000f2a8ed71 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:48.475058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:48.503655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:48.504557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:48.504850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:48.505132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] PATH_INFO: `/v3/users/7249eadb739a42e5b4d211e9be7b5424` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:48.506398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] RBAC: Authorizing `identity:get_user(user_id=7249eadb739a42e5b4d211e9be7b5424)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:48.507898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:48.509562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 81/154] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:48 2022] GET /identity/v3/users/7249eadb739a42e5b4d211e9be7b5424 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:48.515810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:48.657092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9613-4fe7-4347-8fd3-e4808bd87139 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:48.701525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:48.702403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:48.702687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:48.702984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:48.704189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:48.705642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cfc19-441b-4309-99d6-0650d5bc6eeb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:48.707431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 74/155] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:48 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:48.713194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:48.827259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a3b6-86e3-495f-a42d-506662e02700 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:48.874605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:48.875531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:48.875820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:48.876106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/7249eadb739a42e5b4d211e9be7b5424/roles/7e44f86ffd224a0bac4489c48e9da64e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:48.884233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, user_id=7249eadb739a42e5b4d211e9be7b5424, role_id=7e44f86ffd224a0bac4489c48e9da64e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:48.885704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be5dc3-af3c-48a3-902b-f55c83b04359 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:48.978334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 82/156] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:48 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/users/7249eadb739a42e5b4d211e9be7b5424/roles/7e44f86ffd224a0bac4489c48e9da64e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:49.051662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671cf83-5921-444b-928b-69907ed35d65 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.051998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671cf83-5921-444b-928b-69907ed35d65 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.052230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671cf83-5921-444b-928b-69907ed35d65 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.053920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 75/157] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:49.058851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da229d-d670-4afc-b8c5-33082f6ef5d0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.059191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da229d-d670-4afc-b8c5-33082f6ef5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.059478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da229d-d670-4afc-b8c5-33082f6ef5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.083969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-01da229d-d670-4afc-b8c5-33082f6ef5d0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:49.085196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-01da229d-d670-4afc-b8c5-33082f6ef5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:49.128155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:49.184822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 83/158] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:49.240304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671cf83-5921-444b-928b-69907ed35d65 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:49.274828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:49.303911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf861a-3235-4d0f-8e44-670ad3b0a5bc None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.304235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf861a-3235-4d0f-8e44-670ad3b0a5bc None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.304482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf861a-3235-4d0f-8e44-670ad3b0a5bc None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.305865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 84/159] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:49.310111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537b2a-f93c-468c-b426-fb41356c95eb None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.310738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537b2a-f93c-468c-b426-fb41356c95eb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.311326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537b2a-f93c-468c-b426-fb41356c95eb None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.313185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 85/160] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:49.314563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10d27d-d88f-45c5-9f40-452a1461369a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.315107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10d27d-d88f-45c5-9f40-452a1461369a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.315606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10d27d-d88f-45c5-9f40-452a1461369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.348846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ed10d27d-d88f-45c5-9f40-452a1461369a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:49.351387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ed10d27d-d88f-45c5-9f40-452a1461369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:49.405451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671cf83-5921-444b-928b-69907ed35d65 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:49.478416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:49.479257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.479547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.480749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.483080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 86/161] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:49.486058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9006-702c-4145-b973-33632400a63e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.489586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9006-702c-4145-b973-33632400a63e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.489872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9006-702c-4145-b973-33632400a63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.512587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:49.521504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-dc2c9006-702c-4145-b973-33632400a63e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:49.523640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dc2c9006-702c-4145-b973-33632400a63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:49.531603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:49.532595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:49.533424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:49.539348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 76/162] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:24:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:49.540516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:49.676312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 87/163] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:49.678270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c9006-702c-4145-b973-33632400a63e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:49.741360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:49.751251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:49.769922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123650e-52bb-439b-af26-722abeb6277c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:49.841479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:49.842289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.842572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.842848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.865467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:49.873884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:49.874730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:49.875568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:49.882358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c9006-702c-4145-b973-33632400a63e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:49.891521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 77/164] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:49.892450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:49.948217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:49.948943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:49.949236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:49.949518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:49.950281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:49.950616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:49.952045 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:49.963195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 88/165] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:24:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:49.964038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.057024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:50.066704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42387d1-63a2-49aa-a712-067bac5d6816 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.090658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3fe33-2e00-42d6-80ad-483601aa4b85 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.101810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.102497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.102785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.103152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.103920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:50.104233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:50.106729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:50.114878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 89/166] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:50.115714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.145710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.149368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.149885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.150358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.175963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:50.197704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:50.198664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:50.199516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 78/167] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:50.205819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.240826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:50.345996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdd86-aa68-4854-930b-5013a5a33282 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.385291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9933383-1b48-45dd-80c3-264af12c61a8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.416800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.417690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.417971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.418240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.419622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:50.421174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:50.423122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 90/168] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:50.423698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.433070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.435373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.435373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.435373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.439629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:50.441094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:50.442735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 79/169] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:50.449156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.482186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:50.624791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:50.677553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834c016-106e-4972-b159-4bb3de409311 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.689009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaf10-7220-47fb-b115-50eaafa32e73 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:50.730204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.731442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.732011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.733074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.786541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:50.787262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:50.787543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:50.787817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:50.788573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:50.788885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:50.790252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:50.807651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 91/170] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:50.808318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:50.812324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:50.814156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:50.898286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:50.923526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 80/171] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:50.924801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.004186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.033389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4bc70-c75c-4013-87c4-ee7e879aabd4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.040399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bc8-e196-4126-a8dc-91ee873ba679 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.082472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.083770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.084263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.084753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] PATH_INFO: `/v3/users/dffa90611fbe4b429e2703e120177f8e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.092270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa90611fbe4b429e2703e120177f8e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.093978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.095910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 81/172] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/users/dffa90611fbe4b429e2703e120177f8e => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:51.096897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.125584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.126366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.126654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.126945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.186985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.237023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.238432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24537bfe-cde4-496c-964d-dc834c76ade1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.292073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804b3e-83f3-4d05-84f4-9aef2b609f37 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.343055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.343923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.344202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.344488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.345669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.347169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.348872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 82/173] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:51.349644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.372796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.413493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 92/174] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:51.414863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b711-5d35-43c8-a386-320c5d692fd5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.415178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b711-5d35-43c8-a386-320c5d692fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.415460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b711-5d35-43c8-a386-320c5d692fd5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.416767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 93/175] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:51.417430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4b711-5d35-43c8-a386-320c5d692fd5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.475845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484081a3-7e51-42a4-ae38-ade49147fe92 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.492584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.515589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.516270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.516597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.516874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.590906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4b711-5d35-43c8-a386-320c5d692fd5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.591122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.593077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ddbcd-f249-4c2c-8c77-9edbdc2d42f5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.630522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.631525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.632013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.632559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.633597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:51.634093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.635749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.642288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 94/176] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:51.643430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.699264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.700152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 83/177] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:51.701543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed6274-9593-43d8-baf7-8f29ce63cc03 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.701808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed6274-9593-43d8-baf7-8f29ce63cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.702116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed6274-9593-43d8-baf7-8f29ce63cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.720733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5fed6274-9593-43d8-baf7-8f29ce63cc03 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:51.721998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5fed6274-9593-43d8-baf7-8f29ce63cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:51.733758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:51.764286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9c1f-2681-4952-86c5-8b2fc9c79fe1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.797479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 84/178] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:51.798947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b759-119f-4865-85ec-17ea1c18477f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.799241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b759-119f-4865-85ec-17ea1c18477f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.799519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b759-119f-4865-85ec-17ea1c18477f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.800800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 85/179] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:51.801874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d9fd-6417-44b1-93b7-6472fe620b49 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.802154 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d9fd-6417-44b1-93b7-6472fe620b49 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.802428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d9fd-6417-44b1-93b7-6472fe620b49 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.803695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 86/180] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:51.805656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78d9fd-6417-44b1-93b7-6472fe620b49 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.818740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.819653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.819935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.820213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.821456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.822892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebff51-6aeb-4ff3-866d-ea2ceea6f42c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.824870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 95/181] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:51.826575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693b0e8-0d48-467f-8cdd-d66bf982d990 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.826859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693b0e8-0d48-467f-8cdd-d66bf982d990 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.827165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693b0e8-0d48-467f-8cdd-d66bf982d990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.850716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8693b0e8-0d48-467f-8cdd-d66bf982d990 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:51.852276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8693b0e8-0d48-467f-8cdd-d66bf982d990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:51.931692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 96/182] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:51.932843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.933136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.933413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.935886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78d9fd-6417-44b1-93b7-6472fe620b49 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:51.952971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:51.954150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:51.974137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:51.975044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:51.975327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:51.975601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] PATH_INFO: `/v3/projects/e5be27e98a4b4cf78bda910d7f100fea` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:51.976773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] RBAC: Authorizing `identity:get_project(project_id=e5be27e98a4b4cf78bda910d7f100fea)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:51.978230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:51.980296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 87/183] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/projects/e5be27e98a4b4cf78bda910d7f100fea => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:51.980881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:51.998806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.002499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.033964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 97/184] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:52.033964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.106579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf0a4-8f87-4b65-9ef2-029074112d11 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.136087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd00c3-cab0-4ac2-a86f-c54cb2742005 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.142528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.143346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.143633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.143910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.156090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:52.163789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:52.164868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 88/185] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:52.165875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.181500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.182172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.182454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.182738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.183749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:52.184067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.186186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.192783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 98/186] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.193503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.252452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.268699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.300975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de4418-fe06-4e84-8293-c43c7eb5a081 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.341564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.342112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc079e-092f-450c-a1bc-4dfd4b50d74c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.342374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.342660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.342951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.344203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.346669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.352392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 89/187] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.353333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.391517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.392757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.393042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.393319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.394259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=e5be27e98a4b4cf78bda910d7f100fea, user.id=dffa90611fbe4b429e2703e120177f8e) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:52.394602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.396935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.403654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 99/188] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/role_assignments?user.id=dffa90611fbe4b429e2703e120177f8e&scope.project.id=e5be27e98a4b4cf78bda910d7f100fea&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.404665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.536525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f6af3-e935-474f-b522-5468ca0db8b0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.579572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.580550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.580942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.581228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.583041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf157f7-d11b-42fe-9240-34eb322e083a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.584187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.592218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.593225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:52.602587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:52.603577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:52.604379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 90/189] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:52.605191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.627547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.628234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.628514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.628797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.629557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:52.629866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.631287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3e26e-800b-4cde-869d-7e53d8966744 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 100/190] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51583-8c07-405d-89bc-d255e2f08628 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51583-8c07-405d-89bc-d255e2f08628 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51583-8c07-405d-89bc-d255e2f08628 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 101/191] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:52.645275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51583-8c07-405d-89bc-d255e2f08628 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.718623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162f514-6999-4879-87d2-697b88bc688d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.748895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51583-8c07-405d-89bc-d255e2f08628 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.752086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.752749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.753032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.753309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.754016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.755510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0442db-1a7e-460a-b31d-4965d3e0c8c1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.780889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.781562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.781850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.782133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.782887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:52.783234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.784602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.792337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 102/192] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.793067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.822218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.868696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 91/193] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:52 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:52.880526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.880815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.881095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.895770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e364-1402-41a1-b66e-54699d23f1f7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:52.898790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:52.900117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:52.936216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:52.937721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:52.938250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:52.938714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:52.954836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:52.955472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:52.957594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:52.959657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 103/194] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:52.960723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:52.977665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 92/195] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:52.979668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.067086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.084851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf76d2-6597-454d-9b48-b5f64d9e47c4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.092682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd103f3f-9238-4029-aef1-5591c5a25aa0 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.134152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.135109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.135301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.135541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.136854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.137951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.138452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.139240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.141862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.143445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.145690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 104/196] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:53.146474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.213413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.230465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.232902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.288544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305c63f-c1e5-4a6f-971a-7f4f3f1de190 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.329957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.330836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.331241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.331607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.343236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:53.345578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 93/197] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:53.346489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:53.351461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:53.352436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:53.353392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 105/198] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:53.354250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.370700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.455361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.470956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f766274-e37e-407c-9f07-932aa84ccadd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.472873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ded5-d694-439c-8b59-9641410c985f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.512028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.513592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.513896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.514176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.518371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.519370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.519721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.520066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.521318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.522907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.524761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 106/199] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:53.525588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.603501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.605636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.612131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.632936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24ec4-08ab-4db3-862d-a7a15962d9a9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.667963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.668743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.669096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.670025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.670867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:53.671263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.672952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.681443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 107/200] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:53.682323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.717223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.735422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 94/201] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:53.736425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.818301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2dd26-a2a4-47ec-a708-ba714744bf0f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.845568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3a12-bace-4965-9d2e-4d1c7c87c4cf None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:53.863616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.864560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.865239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.866080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.867126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=dd42bbc5c8024e66aa45fc306725ba06) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:53.867619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.869760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.878309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 108/202] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/role_assignments?user.id=dd42bbc5c8024e66aa45fc306725ba06&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:53.880064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.884492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:53.885133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:53.885487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:53.885834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] PATH_INFO: `/v3/users/7249eadb739a42e5b4d211e9be7b5424` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:53.887351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] RBAC: Authorizing `identity:get_user(user_id=7249eadb739a42e5b4d211e9be7b5424)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:53.889075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:53.891643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.895462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 95/203] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/users/7249eadb739a42e5b4d211e9be7b5424 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:53.897103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:53.974575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:53.998857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5c1e-592c-43af-9e0a-12e9c91ded28 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:54.054060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5820d6a-e32b-4b3c-b620-b2ba268cc39a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:54.066881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:54.070329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:54.070630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:54.070911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:54.074977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:54.077031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:54.084218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 109/204] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:54.085152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:54.134765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:54.135887 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:54.136366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:54.136838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:54.201974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:54.213159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:54.219947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:54.231429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30598ed0-78b1-42c5-a134-6c062255dda1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:54.275457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:54.276282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:54.276973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:54.277264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:54.277548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:54.278402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=dd42bbc5c8024e66aa45fc306725ba06) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:54.279366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:54.280935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:54.295659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 110/205] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:54 2022] GET /identity/v3/role_assignments?user.id=dd42bbc5c8024e66aa45fc306725ba06&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:54.326208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 96/206] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:54.331770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:54.494459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272cfa-e099-404d-ba85-59efe1e2ba3b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:54.530010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:54.530884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:54.531202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:54.531486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:54.532679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:54.534135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab6fa0-d45d-42fe-b1bc-32b0ff6c6ef3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:54.541560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 111/207] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:54 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:54.542909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:54.670772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d1cda-6163-4c48-8728-56b7f4e1ff58 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:54.721499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:54.722178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:54.722468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:54.722752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:54.723627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, scope.project.id=9e8c266acd9344289570f2dc0855e7ca, user.id=7249eadb739a42e5b4d211e9be7b5424) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:54.723950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:54.725422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2280a3e-d9df-45c9-a3a4-8e00309c2596 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:54.738119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 97/208] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:54 2022] GET /identity/v3/role_assignments?user.id=7249eadb739a42e5b4d211e9be7b5424&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:55.380749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3097a0-0d66-472e-990f-0023e34da52a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.381137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3097a0-0d66-472e-990f-0023e34da52a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.381314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3097a0-0d66-472e-990f-0023e34da52a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.382968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 112/209] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:55.388421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce4a27-dc17-49a5-8032-b02037fa83d1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.388717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce4a27-dc17-49a5-8032-b02037fa83d1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.388999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce4a27-dc17-49a5-8032-b02037fa83d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.410558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6bce4a27-dc17-49a5-8032-b02037fa83d1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:55.411817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6bce4a27-dc17-49a5-8032-b02037fa83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:55.437448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:55.548782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 98/210] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:55.574449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3097a0-0d66-472e-990f-0023e34da52a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:55.585744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:55.649595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555459-3023-4b77-8651-adce8fa5a3d0 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.649906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555459-3023-4b77-8651-adce8fa5a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.650160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555459-3023-4b77-8651-adce8fa5a3d0 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.651721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 99/211] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:55.656876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.657167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.657445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.675721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:55.677404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:55.690637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3097a0-0d66-472e-990f-0023e34da52a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:55.729872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:55.730790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.731098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.731376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.748548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:55.756218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:55.757617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:55.759618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 113/212] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:55.765373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:55.765507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 100/213] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:55.791145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:55.846525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:55.874186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:55.881594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cb759-2db7-45a0-95ef-fa8f363720bc None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:55.897105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9621-d5a1-4626-8fe4-079fac1be99f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:55.918959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:55.919539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.919828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.920911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:55.921173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:55.922618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:55.935510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 114/214] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:55.937857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:55.938966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:55.940010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:55.940491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:55.941043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:55.943019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:55.944729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:55.951711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 101/215] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:55.952743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.044858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41946bb0-a376-4c1b-b89f-33b34f22b283 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.057789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f3ef-51ec-4933-ad89-54431b14956e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.074128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.083453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.085864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.085864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.085864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] PATH_INFO: `/v3/users/6dc52b319fa34ed7bf0308dfe6747048` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.097218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.098317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.099062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.099463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] RBAC: Authorizing `identity:get_user(user_id=6dc52b319fa34ed7bf0308dfe6747048)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.099788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.100759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.100949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.102626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 115/216] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:55 2022] GET /identity/v3/users/6dc52b319fa34ed7bf0308dfe6747048 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:56.102915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc87ad-ffc0-415c-bcd0-4f13a4c3a55e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.108556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.177512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 102/217] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:55 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:56.229748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58b84-cb8d-48d2-b8c8-b9cd33ba9b5e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.269892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.270631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.270933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.271215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.339300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.355395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.356917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.368629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48017be-a73f-4957-84cc-eca3363e807c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.368912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48017be-a73f-4957-84cc-eca3363e807c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.369196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48017be-a73f-4957-84cc-eca3363e807c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.372177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 103/218] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:56.379971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b745-d491-43da-8f20-889b3f929646 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.380259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b745-d491-43da-8f20-889b3f929646 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.380539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594b745-d491-43da-8f20-889b3f929646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.387333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.401517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0594b745-d491-43da-8f20-889b3f929646 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:56.403064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0594b745-d491-43da-8f20-889b3f929646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:56.485134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 116/219] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:24:56.491163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.493633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 104/220] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:56.517920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594b745-d491-43da-8f20-889b3f929646 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.594304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.618099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f476e-ca38-4cb4-bdb3-401b50a5d9cd None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.649065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594b745-d491-43da-8f20-889b3f929646 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.668767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.669726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.670041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.670457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.672006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.673462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f49c85-e7de-48c6-9ba4-d1e7933e290a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.675669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 117/221] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:56.677076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f4d8-1ae4-4846-a164-f5026e4a7231 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.677365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f4d8-1ae4-4846-a164-f5026e4a7231 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.677644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7f4d8-1ae4-4846-a164-f5026e4a7231 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.678954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 118/222] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:56.687084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.687084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.687084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.688500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.693507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.694318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.694605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.694882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.705730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:56.706548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:56.706953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:56.714358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:56.715322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:56.716129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 105/223] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:24:56.716834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.786558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 119/224] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:56.787426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.839802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f49a16-9b9a-406c-a902-400357549352 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.885343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.886655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:56.887071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.888780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.889278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.890344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=6dc52b319fa34ed7bf0308dfe6747048) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:56.890945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.893416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.901858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 106/225] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/role_assignments?user.id=6dc52b319fa34ed7bf0308dfe6747048&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:56.903806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.903915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1653144-4d39-4fe8-a577-282bc346b238 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:56.943499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:56.944288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:56.944577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:56.944849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:56.945614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:56.945921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:56.947324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:56.957904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 120/226] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:56.963520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:56.981644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.036528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba48a29-a215-4c82-9d28-5e5972703455 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.075360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.076366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.076737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.077118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.088739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:57.096195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7921-a719-430d-9da5-36047b1ce404 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:57.097484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:57.098466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:24:57.099300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:57.099900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 107/227] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:57.112105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.141009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.142302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.142735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.143183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.146646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.149815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.156253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 121/228] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:56 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:57.159706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.183776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.272381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cab6-a08d-47f8-ae83-d4de7f23affb None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.307390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.308279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.308663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.309042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.309961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:57.310392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.312174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.317946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.321315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 108/229] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:24:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:57.327131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.350797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d24935-fe83-4c69-a922-74be40a2f8c3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.456223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.457112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.457405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.457687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.463685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.464086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3031f5-dd98-4f85-a9ba-fb7d52557d13 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.465162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.467147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 122/230] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:57 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:57.479845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.505714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.506640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.506946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.507229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.508542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.510578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.512274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 109/231] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:57 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:57.519825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.571292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.626591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea72fd-740f-4dcb-b502-52174dc0e88f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.666097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.666988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.667278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.667552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.668956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.670444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.676942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 110/232] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:24:57 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:57.686244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:57.699342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.731526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066f8ae-951e-4ba4-bc7e-9c24bdf8301e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.821167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e05591-f5c0-4fcc-bb6d-d20d6baca8f3 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:57.851440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:57.859824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.860921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.861213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.861495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/dd42bbc5c8024e66aa45fc306725ba06/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.864030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, user_id=dd42bbc5c8024e66aa45fc306725ba06, role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.865556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6bd43-7b5b-414f-8834-df178ee77c9c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.871705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:57.873199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:57.873516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:57.873787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/dd42bbc5c8024e66aa45fc306725ba06/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:57.880455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, user_id=dd42bbc5c8024e66aa45fc306725ba06, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:57.882206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e515-a199-4cbf-9f72-3e81d3ff19e0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:57.949744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 111/233] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:57 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/dd42bbc5c8024e66aa45fc306725ba06/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:57.972586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 123/234] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:24:57 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/dd42bbc5c8024e66aa45fc306725ba06/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:24:58.881031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9a04-0a3e-4cee-bfc4-72933d00cb58 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:58.881555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9a04-0a3e-4cee-bfc4-72933d00cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:58.881631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9a04-0a3e-4cee-bfc4-72933d00cb58 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:58.882985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 112/235] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:58.888838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095afc2-b48e-4cc3-9ff3-89be88a5c26e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:58.889256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095afc2-b48e-4cc3-9ff3-89be88a5c26e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:58.889420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095afc2-b48e-4cc3-9ff3-89be88a5c26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:58.911036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c095afc2-b48e-4cc3-9ff3-89be88a5c26e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:58.912239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c095afc2-b48e-4cc3-9ff3-89be88a5c26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:58.998057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 124/236] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:59.040490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d9a04-0a3e-4cee-bfc4-72933d00cb58 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.068259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5e1c-0987-4d06-a986-31a14619554c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.068578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5e1c-0987-4d06-a986-31a14619554c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.068825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5e1c-0987-4d06-a986-31a14619554c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.070347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 125/237] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:24:59.075239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.075631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.076372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.104041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:59.105260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:59.257159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 126/238] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:24:59.262493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d9a04-0a3e-4cee-bfc4-72933d00cb58 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.282087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.284659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:59.356301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.357663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.357943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.358334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.391918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:59.407796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:59.408706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:24:59.409494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 113/239] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:24:59.412541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eb948-1c8e-414f-917b-bc696619352c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.415085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.458446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.459205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.459502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.459779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.460976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:59.462433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7a034-9c8c-45ff-afde-05de00c3aed4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:59.464427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 127/240] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:59.465623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d994129-e31a-489d-8917-ec5d39ea17e5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.470210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d994129-e31a-489d-8917-ec5d39ea17e5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.470210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d994129-e31a-489d-8917-ec5d39ea17e5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.471683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 128/241] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:24:59.472700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d994129-e31a-489d-8917-ec5d39ea17e5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.497198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:59.557034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:59.561738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d4b94-28da-4712-ab9c-dc219afae88e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.590840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d994129-e31a-489d-8917-ec5d39ea17e5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.597624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def15058-3960-4622-a252-58cc4b46570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.598299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def15058-3960-4622-a252-58cc4b46570f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.598588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def15058-3960-4622-a252-58cc4b46570f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.598870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def15058-3960-4622-a252-58cc4b46570f None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.599768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def15058-3960-4622-a252-58cc4b46570f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:24:59.600082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def15058-3960-4622-a252-58cc4b46570f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:59.601588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def15058-3960-4622-a252-58cc4b46570f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:59.612748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 114/242] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:24:59.618418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.618719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.619055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.636993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.638044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.638562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.638798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:24:59.639264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.640073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:24:59.640451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:59.642406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:59.720254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 115/243] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:24:59.722433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 129/244] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:24:59 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:24:59.723780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.744443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.757325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:59.833844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c745-c494-4d37-a2d4-a05cea03b20d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.858882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c4a0d-4c46-43c9-bc8f-93b461e142d5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:24:59.870041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.870946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.871256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.871758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] PATH_INFO: `/v3/users/343fead7ea6c46289e07fe4664160e6c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.886070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] RBAC: Authorizing `identity:get_user(user_id=343fead7ea6c46289e07fe4664160e6c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:24:59.887592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:24:59.893906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 116/245] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/users/343fead7ea6c46289e07fe4664160e6c => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:24:59.896015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:24:59.896496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.907030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:24:59.908176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:24:59.908817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:24:59.909213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:24:59.921838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:24:59.931806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:24:59.932749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:24:59.933633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:24:59.940765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 130/246] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:24:59.950121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:24:59.995605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.023537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561319-4416-4160-b2fa-d694ef812058 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.068288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.069048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.069335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.069608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.087429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc140b-64ca-4430-b42c-267fe818f927 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.124970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.125652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.125938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.126217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.127005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:00.127318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.128713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.137595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 131/247] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:00.144143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.152903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.154471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.210708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.261084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.280559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa567-6dd6-4396-8ada-a2a29177b3c9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.282294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 117/248] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:00.291078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.346667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.347895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.348336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.348675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] PATH_INFO: `/v3/users/6dc52b319fa34ed7bf0308dfe6747048` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.350229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] RBAC: Authorizing `identity:get_user(user_id=6dc52b319fa34ed7bf0308dfe6747048)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.352582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.354975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 132/249] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:00 2022] GET /identity/v3/users/6dc52b319fa34ed7bf0308dfe6747048 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:00.360911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.416899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203272c-bb21-40ea-8f7d-0cd33acf9afd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.454143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.455412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.455900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.456394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.457844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.459571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.461838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 118/250] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:00 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:00.469074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.490478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03912-81fc-4b3e-aec3-a61fb6060846 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.515400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.543142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.544109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.545034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.545986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.546818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.550035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.551618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.554915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 133/251] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:00 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:00.563271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.617506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3026f-4caa-4a34-998b-a2c57bea8508 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.670437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.672216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.672505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.672784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.674398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=343fead7ea6c46289e07fe4664160e6c) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:00.674727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.676249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b8e-fe56-46a3-8a0e-2def242c6cd4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.686089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf067c-f15a-467b-9067-6a90c919a2b9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:00.690329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 119/252] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:00 2022] GET /identity/v3/role_assignments?user.id=343fead7ea6c46289e07fe4664160e6c&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:00.692788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcdf49-f52b-424d-a627-04b80407665b None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.693079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcdf49-f52b-424d-a627-04b80407665b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.693358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcdf49-f52b-424d-a627-04b80407665b None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.700044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 120/253] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:00.705734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab5e1-4080-4165-904b-54f747c1a902 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.706050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab5e1-4080-4165-904b-54f747c1a902 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.706301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ab5e1-4080-4165-904b-54f747c1a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.735353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-810ab5e1-4080-4165-904b-54f747c1a902 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:00.736656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-810ab5e1-4080-4165-904b-54f747c1a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:00.762634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:00.763671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.763960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.764242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/6dc52b319fa34ed7bf0308dfe6747048/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.770157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, user_id=6dc52b319fa34ed7bf0308dfe6747048, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:00.771581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:00.832961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 121/254] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:00.834329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41753-228c-442c-94f1-ae9050ee0912 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.834619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41753-228c-442c-94f1-ae9050ee0912 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.834897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41753-228c-442c-94f1-ae9050ee0912 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.836487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 122/255] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:00.842485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:00.842776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:00.843090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:00.860978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:00.869032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 134/256] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:00 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/6dc52b319fa34ed7bf0308dfe6747048/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:25:00.869215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:00.871779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:00.937564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.963023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:00.980931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 123/257] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:01.016503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.067493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c18e3-fa8b-4240-9641-951273afb3c2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.143861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.144690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.144974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.145252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.162768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:01.176444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:01.177394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:01.178196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:01.179323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 135/258] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:01.189101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.246323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bd789-4c72-4a11-a88d-85a9bb2ffe8c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.373851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.376241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c34f8c-01ac-4367-9d57-4cbfbfad0e8e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.378578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.379277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.380342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:01.381166 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.402891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:01.417285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:01.418217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:01.419061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:01.420656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 124/259] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:01.425828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.447929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.448927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.449389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.449839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.451467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:01.453506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:01.455105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:01.474633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 136/260] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:01.480361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.498956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:01.626907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cccf3e-e94a-47f4-842d-4af210e8ab36 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.680632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef9297-8578-4b63-9470-e1c6815bcaba None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.691820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.692590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.692875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.693151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.694019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:01.694332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:01.695756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:01.710966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 125/261] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:01.717700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.739504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.740717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.741197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.741665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.743365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:01.746501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:01.751064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 137/262] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:01.756235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.812985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:01.848628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dca9a-83d6-4365-8e16-30612a9225ef None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:01.859303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:01.888855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:01.890206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:01.890708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:01.891256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] PATH_INFO: `/v3/users/dd42bbc5c8024e66aa45fc306725ba06` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:01.892924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] RBAC: Authorizing `identity:get_user(user_id=dd42bbc5c8024e66aa45fc306725ba06)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:01.894595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:01.897002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 126/263] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/users/dd42bbc5c8024e66aa45fc306725ba06 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:01.904612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:01.952919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149605e7-f76a-4ddc-8f39-af69c3c2d733 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.015675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad0f-e2d0-4473-bda4-5a86aeaa1566 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.035675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.036354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.036648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.036932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.053591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.054557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.055086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.055567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.099351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:02.122420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.123987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.135674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.137393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.173661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:02.246068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 127/264] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:02.251717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:02.269967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 138/265] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:02.275388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:02.379244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:02.391957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db06573-fcd2-46fa-9803-e6dcf5eddd92 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.431455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.432977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.433460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.434060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.436052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.437970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.440414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 139/266] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:02 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:02.446798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:02.453301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:02.461019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2cbf-4d69-453a-989f-489fb6cfc794 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.532485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.533349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.533631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.533907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.536250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.537692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.539594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 128/267] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:02 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:02.546295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:02.658906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3780a-0c51-4b78-9144-63482431ddad None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.719234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.719912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.720201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.720481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.721333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=dd42bbc5c8024e66aa45fc306725ba06) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:02.721645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.723118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d40eca-1977-4c0b-9eb6-0b6c626ff49d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.737324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 140/268] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:02 2022] GET /identity/v3/role_assignments?user.id=dd42bbc5c8024e66aa45fc306725ba06&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:02.739301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e642f5-4fa9-44fa-9227-881d9957cc64 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:02.784710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:02.785401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:02.785748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:02.786033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:02.787001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=dd42bbc5c8024e66aa45fc306725ba06) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:02.787383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:02.788885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2a49-a33d-4a3f-904e-94d76e3ab4b4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:02.791024 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:02.800068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 129/269] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:02 2022] GET /identity/v3/role_assignments?user.id=dd42bbc5c8024e66aa45fc306725ba06&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:03.353632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469333-d0a6-486b-90b1-2186385eac96 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.354147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469333-d0a6-486b-90b1-2186385eac96 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.354250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469333-d0a6-486b-90b1-2186385eac96 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.355715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 141/270] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:03.357781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b693f0-1974-4bbd-9e4b-da635c52edd4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.358189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b693f0-1974-4bbd-9e4b-da635c52edd4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.358341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b693f0-1974-4bbd-9e4b-da635c52edd4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.359739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 130/271] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:03.361777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.362064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.362343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.370098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.370403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.370652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.389326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:03.390854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:03.397658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:03.397658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:03.479734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:03.490969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 142/272] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:03.507972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 131/273] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:03.522345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:03.538424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:03.632270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f41d2-9e5e-4cc6-aa62-80c4a52a2c7a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:03.664875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e20f89-7ab8-4c32-9d1d-d03a6e724a6e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:03.674668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:03.675712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.676066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.676413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.688079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:03.696473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:03.697389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:03.698212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:03.703435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:03.704153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 143/274] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:03.705686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:03.710512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:03.716772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.717219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.717879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.721727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:03.723658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:03.726137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 132/275] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:03.733343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:03.775337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:03.825094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23460190-da2a-4153-a2c8-3112fc646c3f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:03.851934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05349-a13c-4b84-8f3d-63ccce83063d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:03.867521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:03.868932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.869454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.869958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.870908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:03.871865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:03.873743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85d246-e025-4b04-9c54-55c3d120ece6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:03.886838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:03.886838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.886838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.886838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.887240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:03.887308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 144/276] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:03.888768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda689f-de78-4f21-817e-18d255b27bea None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:03.889021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb1c3-3356-492e-90cb-3ebf78b1b0a0 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.889310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb1c3-3356-492e-90cb-3ebf78b1b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.889586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb1c3-3356-492e-90cb-3ebf78b1b0a0 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:03.890971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 145/277] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:03.895706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eb1c3-3356-492e-90cb-3ebf78b1b0a0 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:03.983835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 133/278] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:03 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:03.985628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440efd-08c7-42d0-9438-93d51b1bf90f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:03.986003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440efd-08c7-42d0-9438-93d51b1bf90f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:03.986360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23440efd-08c7-42d0-9438-93d51b1bf90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.006189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-23440efd-08c7-42d0-9438-93d51b1bf90f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:04.007872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-23440efd-08c7-42d0-9438-93d51b1bf90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:04.016710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eb1c3-3356-492e-90cb-3ebf78b1b0a0 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.052252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.072242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.073154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.073656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.073790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.073790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] PATH_INFO: `/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.093156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] RBAC: Authorizing `identity:get_user(user_id=f82ad9f1cae04fc288a0afc9a3ceefa2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.094865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836cba8-2efd-4309-9939-ea2831e00e0b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.097301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 146/279] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:03 2022] GET /identity/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:04.098227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0612-62e0-4e7e-aeb0-66b6187a5d9d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.098514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0612-62e0-4e7e-aeb0-66b6187a5d9d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.098791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0612-62e0-4e7e-aeb0-66b6187a5d9d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.100263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 147/280] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:04.104494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0612-62e0-4e7e-aeb0-66b6187a5d9d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.108889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 134/281] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:04.110323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.111102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.111391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.131875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:04.133104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:04.224134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 135/282] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:04.224134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.226084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0612-62e0-4e7e-aeb0-66b6187a5d9d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.271026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.272449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.272737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.273018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.273863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, user.id=f82ad9f1cae04fc288a0afc9a3ceefa2, scope.system=all) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:04.274184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.275827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.284810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 148/283] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/role_assignments?user.id=f82ad9f1cae04fc288a0afc9a3ceefa2&scope.system=all&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:04.285579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.355370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.360565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b9e23-049d-43a0-8c78-a410af2210f0 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.377350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.409986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.410949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.411348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.411698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.424791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:04.431293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f08a4-b2b3-42e6-bf38-52330c5842a7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:04.432705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:04.433658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:04.434447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:04.437960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 136/284] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:04.442498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.477901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.478556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.478845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.479150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.495528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:04.504514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:04.505501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:04.506327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 149/285] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:04.513562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.574146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7a3de-9479-4b62-981a-bf630ae326aa None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.619671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.620783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.621277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.621851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.622831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:04.623369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.625164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.627761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.638685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72289583-4de7-4123-846e-8c9113d2b21b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.643959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 137/286] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:04.647861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.697585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.706543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.707536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.716092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.716092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.716092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:04.716092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.716092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.720965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 150/287] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:04.726464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.835141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a7e2-20f4-431d-bd3c-73ee84d7d1d7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.857067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607bde2-169d-459d-9bb3-60866e6bcc28 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:04.873144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.874035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.874320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.874594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] PATH_INFO: `/v3/users/6dc52b319fa34ed7bf0308dfe6747048` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.876022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=6dc52b319fa34ed7bf0308dfe6747048)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.877402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.879365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 151/288] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/users/6dc52b319fa34ed7bf0308dfe6747048 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:04.884711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.917390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:04.923425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:04.924309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:04.924580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:04.924854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] PATH_INFO: `/v3/users/343fead7ea6c46289e07fe4664160e6c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:04.926058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] RBAC: Authorizing `identity:get_user(user_id=343fead7ea6c46289e07fe4664160e6c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:04.927554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:04.929195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 138/289] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/users/343fead7ea6c46289e07fe4664160e6c => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:04.936085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:04.989543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f99d8-7688-4fd2-af58-a0df1ed3c1d8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:05.036364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:05.037026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:05.037308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:05.037586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:05.108221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59703f30-813c-4ab3-9246-7a40cc7524b0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:05.109691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:05.148925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:05.150333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:05.150627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:05.150909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:05.152157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:05.153871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:05.156099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 139/290] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:05.164217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:05.220002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:05.221465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:05.284183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd10ef-8eec-4b11-945e-bec9bab756e0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:05.300588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:05.335148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:05.336114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:05.336407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:05.336684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/343fead7ea6c46289e07fe4664160e6c/roles/7e44f86ffd224a0bac4489c48e9da64e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:05.342113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, user_id=343fead7ea6c46289e07fe4664160e6c, role_id=7e44f86ffd224a0bac4489c48e9da64e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:05.343642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:05.407205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 152/291] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:05.412968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:05.453803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 140/292] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:05 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/users/343fead7ea6c46289e07fe4664160e6c/roles/7e44f86ffd224a0bac4489c48e9da64e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:05.532078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd614-cd7f-414b-bb99-6770c5f53870 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:05.567565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:05.571544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:05.572464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:05.572772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:05.573033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:05.574496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:05.576142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d3619-e47b-44f5-ad7d-391fda4cce4e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:05.577925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 153/293] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:05 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:05.583991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:05.626645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:05.719687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0075d9-ff74-465e-b3f9-e57d89e7f4c6 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:05.756300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:05.756958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:05.757244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:05.757519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:05.758363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=6dc52b319fa34ed7bf0308dfe6747048) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:05.758678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:05.760159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a310e0-7e8c-4ddc-ab59-acf8a220735a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:05.766435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 141/294] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:05 2022] GET /identity/v3/role_assignments?user.id=6dc52b319fa34ed7bf0308dfe6747048&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:06.610765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edd74-7b57-45e0-993f-12be2f2dce9b None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.611309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edd74-7b57-45e0-993f-12be2f2dce9b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.611386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edd74-7b57-45e0-993f-12be2f2dce9b None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.612858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 154/295] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:06.617825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.618223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.618384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.636662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:06.638465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:06.706159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08bec0-2cd8-40b5-9c49-54d5fc48a042 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.706473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08bec0-2cd8-40b5-9c49-54d5fc48a042 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.706723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08bec0-2cd8-40b5-9c49-54d5fc48a042 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.708498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 155/296] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:06.713391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 142/297] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:06.714407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745eaf-d725-4212-99c5-91260450d3b6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.714666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745eaf-d725-4212-99c5-91260450d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.714985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745eaf-d725-4212-99c5-91260450d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.736938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:06.740900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d5745eaf-d725-4212-99c5-91260450d3b6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:06.742487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d5745eaf-d725-4212-99c5-91260450d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:06.763832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:06.814610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 156/298] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:06.816311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118e0a-c72d-41cf-8c1d-664817462147 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.816669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118e0a-c72d-41cf-8c1d-664817462147 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.817040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118e0a-c72d-41cf-8c1d-664817462147 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.818534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 157/299] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:06.824456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.825076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.825501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.843208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:06.844539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:06.849083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4641874-6b68-4064-a741-fb9a89e2c994 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:06.886049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:06.887146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:06.887431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:06.887707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:06.898352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:06.906136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:06.907105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:06.907909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 143/300] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:06.908578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:06.924692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 158/301] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:06.925399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:06.993498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.002846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.018333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c5062-dfe9-4171-9e45-449dd7d18f92 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.036228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cf8ee-7fce-4720-905a-d60fd60b260e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.053095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.053884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.054169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.054440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.055653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.057376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.059528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 144/302] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:07.060893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.079344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.080023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.080304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.080583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.081330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:07.081638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.083027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.095764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 159/303] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:07.095764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.169916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19577f3e-4519-4f44-b1cc-7d8ffd384acc None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.209103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.209563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba7806-cfe4-4502-92c5-6b0f48dd46e3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.210022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.210307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.210595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.221184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:07.228797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:07.229677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:07.230485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 145/304] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:07.232708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.233903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.246850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.247750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.248033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.248359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] PATH_INFO: `/v3/groups` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.249046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.250229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.251397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.323012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 160/305] 10.222.0.36 () {60 vars in 1134 bytes} [Thu Jun 16 19:25:07 2022] POST /identity/v3/groups => generated 229 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:07.323952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.344770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbfe3d-e561-452c-b640-309687c9343f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.395190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.396200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.396594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.397235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] PATH_INFO: `/v3/users/c91184697c3e47de94b21656b1cb97db` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.417483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] RBAC: Authorizing `identity:get_user(user_id=c91184697c3e47de94b21656b1cb97db)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.420211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.423251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 146/306] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:07 2022] GET /identity/v3/users/c91184697c3e47de94b21656b1cb97db => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:07.428985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.478368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43161-f73e-4e1c-a716-89d9e48f5542 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.506814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.524123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.524926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.525218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.525506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.526276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:07.526591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.528036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.541721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 161/307] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:07.542819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.545495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99471ed-5f88-4768-903d-7f7d05e554d8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.582074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.582734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.583055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.583341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.584202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, user.id=c91184697c3e47de94b21656b1cb97db, scope.system=all) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:07.584510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.585957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.593351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 147/308] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Jun 16 19:25:07 2022] GET /identity/v3/role_assignments?user.id=c91184697c3e47de94b21656b1cb97db&scope.system=all&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:07.599407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.661448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dacd1-bc3c-4a83-9411-bc2a40b7dfec None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.700112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.701535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.701827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.702102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] PATH_INFO: `/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.703433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] RBAC: Authorizing `identity:get_user(user_id=f82ad9f1cae04fc288a0afc9a3ceefa2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.704908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a8ca9-1250-4757-af5e-71f4ca301681 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.706996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 162/309] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:07 2022] GET /identity/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:07.712839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:07.826581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef067-5f6e-441c-8d3f-1755a76f3f41 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:07.865769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:07.867073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.867576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.868161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] PATH_INFO: `/v3/system/users/f82ad9f1cae04fc288a0afc9a3ceefa2/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.869860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f82ad9f1cae04fc288a0afc9a3ceefa2, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:07.871525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:07.873809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:07.900918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53ad-ffaf-474c-ad95-a7a854ffefc1 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.901245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53ad-ffaf-474c-ad95-a7a854ffefc1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.901559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da53ad-ffaf-474c-ad95-a7a854ffefc1 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.903299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 163/310] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:07.908993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:07.909281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:07.909559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:07.927222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:07.928403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:08.005422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 148/311] 10.222.0.36 () {58 vars in 1334 bytes} [Thu Jun 16 19:25:07 2022] PUT /identity/v3/system/users/f82ad9f1cae04fc288a0afc9a3ceefa2/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:08.007682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 164/312] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:08.033017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:08.153626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaacc7-57a1-4c50-bd61-0e0d3ef88b67 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:08.191378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:08.192186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:08.192474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:08.192761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:08.204304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:08.212157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:08.213051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:08.213870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 149/313] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:08.220391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:08.262745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:08.324558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd2da6-dd1f-44aa-bd5c-3534acbdbc9f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:08.359736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:08.360726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:08.361150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:08.361569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:08.362463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:08.362939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:08.364547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:08.374235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 165/314] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:08.379590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:08.441333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:08.483052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d85ec-6f49-4f3e-b871-f5fe63857182 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:08.519130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:08.519966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:08.520252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:08.520535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] PATH_INFO: `/v3/users/343fead7ea6c46289e07fe4664160e6c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:08.521968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] RBAC: Authorizing `identity:get_user(user_id=343fead7ea6c46289e07fe4664160e6c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:08.523539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:08.525518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 150/315] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:08 2022] GET /identity/v3/users/343fead7ea6c46289e07fe4664160e6c => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:08.531159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:08.636209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fe2-55c6-4526-aa0a-3aa0f609555e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:08.637716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:08.671375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:08.672111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:08.672315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:08.672598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:08.744862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:08.746373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:08.850861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 166/316] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:08.862982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:08.962913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591e689-983e-4dbd-80db-91119a291c54 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:08.989484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:08.998283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:08.999167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:08.999455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:08.999737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.001023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:09.009364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e371de-6bf3-4112-9c37-0524ebe2f441 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:09.009364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 151/317] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:08 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:09.010691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:09.012918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:09.115711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355d5d-2490-4eb8-aeda-0087ff0af54f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:09.150952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:09.151635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.151918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.152195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.153032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, scope.project.id=1d142bb142314799b8554ac8d6275c5d, user.id=343fead7ea6c46289e07fe4664160e6c) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:09.153348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:09.155421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a697e-4feb-433f-9ab9-d7f709b2d77d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:09.161704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 167/318] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:09 2022] GET /identity/v3/role_assignments?user.id=343fead7ea6c46289e07fe4664160e6c&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:09.572233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c55ac-4c0d-4519-b8cd-1a5a8cfa3db2 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.572609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c55ac-4c0d-4519-b8cd-1a5a8cfa3db2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.572848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c55ac-4c0d-4519-b8cd-1a5a8cfa3db2 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.574313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 152/319] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:09.579998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695124-0b58-410d-bcc0-4faafe97be34 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.580298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695124-0b58-410d-bcc0-4faafe97be34 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.580580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695124-0b58-410d-bcc0-4faafe97be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.596916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-33695124-0b58-410d-bcc0-4faafe97be34 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:09.598103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-33695124-0b58-410d-bcc0-4faafe97be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:09.655084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:09.662379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 168/320] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:09.684449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c55ac-4c0d-4519-b8cd-1a5a8cfa3db2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:09.761929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077633b7-dd6c-47e0-b803-cb3c4763c433 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.762484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077633b7-dd6c-47e0-b803-cb3c4763c433 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.762484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077633b7-dd6c-47e0-b803-cb3c4763c433 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.764700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 169/321] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:09.769389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.769696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.769958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.787823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:09.789017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:09.795995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c55ac-4c0d-4519-b8cd-1a5a8cfa3db2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:09.833555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:09.834627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.835119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.835568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.836895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:09.838496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:09.840492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 153/322] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:09.845997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:09.866968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 170/323] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:09.887104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:09.912604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:09.952800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b21b8-881a-40c4-85b3-12397de9fcf0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:09.986174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:09.986863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:09.987180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:09.987465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] PATH_INFO: `/v3/groups` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:09.988153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:09.989584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:09.991251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae2f2e-8a67-445b-b306-f9e335d1268b None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.017463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:10.025894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.026696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.027015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.027296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.037063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:10.045261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:10.046155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:10.046977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 171/324] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:10.052242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.091377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 154/325] 10.222.0.36 () {60 vars in 1134 bytes} [Thu Jun 16 19:25:09 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:25:10.158378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abee634-5863-4e85-aceb-74703c034176 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.202165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.202857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.203183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.203463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.204236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:10.204548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:10.205946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f237f-aa8a-4f86-80ca-634d62db1fdf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:10.215500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 172/326] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:10.221671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.301675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72834-74d9-4bdf-852f-8e124d13c696 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.301966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72834-74d9-4bdf-852f-8e124d13c696 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.302252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72834-74d9-4bdf-852f-8e124d13c696 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.305506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 173/327] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:10.309604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.309604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.309604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.328796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:10.329988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:10.335367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852c640-1835-4c8d-83bc-ed898f3030b5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.361642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:10.377182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.379286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.379286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.379286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] PATH_INFO: `/v3/users/c91184697c3e47de94b21656b1cb97db` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.380437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] RBAC: Authorizing `identity:get_user(user_id=c91184697c3e47de94b21656b1cb97db)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:10.381762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:10.383179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 155/328] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:10 2022] GET /identity/v3/users/c91184697c3e47de94b21656b1cb97db => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:10.383911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:10.389177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.413383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 174/329] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:10.440782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.550783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b833b8-bd46-4048-8b77-2d4d1a1dfb3d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.596865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.597730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.598018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.598297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] PATH_INFO: `/v3/system/users/c91184697c3e47de94b21656b1cb97db/roles/7e44f86ffd224a0bac4489c48e9da64e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.598429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9416059-d36a-42fd-ba24-cec464fde6e0 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.600158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c91184697c3e47de94b21656b1cb97db, role_id=7e44f86ffd224a0bac4489c48e9da64e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:10.602541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:10.638613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.639401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.639676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.639958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.650559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:10.657616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 156/330] 10.222.0.36 () {58 vars in 1334 bytes} [Thu Jun 16 19:25:10 2022] PUT /identity/v3/system/users/c91184697c3e47de94b21656b1cb97db/roles/7e44f86ffd224a0bac4489c48e9da64e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:10.658545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:10.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:10.660271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 175/331] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:10.669764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.743726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:10.791458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e4ad-7520-4a0f-b495-040d65584a99 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:10.832794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:10.833485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:10.833769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:10.834044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:10.834814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:10.835153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:10.836578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:10.846095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 157/332] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:10.851610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:10.896962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:10.964048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c56f7e-2450-4bbc-b6d8-c151ede7b903 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:11.006531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:11.007455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:11.007733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:11.008012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] PATH_INFO: `/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:11.009223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] RBAC: Authorizing `identity:get_user(user_id=f82ad9f1cae04fc288a0afc9a3ceefa2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:11.010672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd9ca-de77-4596-9fb0-8e2cd45a4d5d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:11.012349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 176/333] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:10 2022] GET /identity/v3/users/f82ad9f1cae04fc288a0afc9a3ceefa2 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:11.018225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:11.189214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe091-259b-4c46-8511-e74d36c6426d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:11.217288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:11.251648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:11.252312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:11.252597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:11.252874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:11.253715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] RBAC: Adding query filter params (role.id=c67c387f18884b4081e6a7754b8319a6, user.id=f82ad9f1cae04fc288a0afc9a3ceefa2, scope.system=all) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:11.254028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:11.255552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14688-afc7-4da5-99bf-e8db139001e6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:11.270852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 158/334] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Jun 16 19:25:11 2022] GET /identity/v3/role_assignments?user.id=f82ad9f1cae04fc288a0afc9a3ceefa2&scope.system=all&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 510 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:12.979021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28949-bf15-46de-ac8c-d013b7147798 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:12.979021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28949-bf15-46de-ac8c-d013b7147798 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:12.979021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28949-bf15-46de-ac8c-d013b7147798 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:12.980092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 177/335] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:12.986066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254f32-852c-4c1a-856a-3e74d71ba127 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:12.986483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254f32-852c-4c1a-856a-3e74d71ba127 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:12.986675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254f32-852c-4c1a-856a-3e74d71ba127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.007989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b3254f32-852c-4c1a-856a-3e74d71ba127 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:13.009194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b3254f32-852c-4c1a-856a-3e74d71ba127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:13.095889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 159/336] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:13.121618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28949-bf15-46de-ac8c-d013b7147798 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:13.293147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28949-bf15-46de-ac8c-d013b7147798 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:13.305035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:13.339750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c1f2-dd09-49c6-b21a-e455fc5e1c72 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.340057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c1f2-dd09-49c6-b21a-e455fc5e1c72 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.340326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c1f2-dd09-49c6-b21a-e455fc5e1c72 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.341717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:13.342005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 160/337] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:13.343004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.343487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.343973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.356173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:13.359065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.359357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.359635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:13.368402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:13.369315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:13.370108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:13.376244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 178/338] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:13.384742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:13.393124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:13.394314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:13.522863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 161/339] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:13.543803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525fc1-8b8b-47f0-936d-14bc25df8314 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:13.556751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:13.617528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:13.617718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:13.618686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.618686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.620948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.620948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:13.620948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:13.621754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:13.640587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 179/340] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:13.646727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:13.711959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:13.718278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff314bc-8158-40c7-bc18-03ed477e44b1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:13.785882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:13.787118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.787657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.788133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.811733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-90feacdd-6a3f-4b63-959f-131a9279aa2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:13.825982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:13.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:13.827731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:13.833880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d846-8249-4bd6-ace9-c0c22c65d135 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:13.835513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 162/341] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:13.838944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b658c-19ce-4bb6-bf95-bb4989cf97db None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.839466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b658c-19ce-4bb6-bf95-bb4989cf97db None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.839968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b658c-19ce-4bb6-bf95-bb4989cf97db None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.852174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 163/342] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:13.853312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b658c-19ce-4bb6-bf95-bb4989cf97db None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:13.911628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:13.912965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:13.913455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:13.913937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:13.988246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.004167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b658c-19ce-4bb6-bf95-bb4989cf97db None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.015419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.017571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd581bf-752e-4984-b7fd-83e474167b96 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.054603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.073393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.074391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.074876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.075426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.076493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:14.077009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.078595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dfcbd-b67d-49f4-975f-3e61ac1a4dd0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.089811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 164/343] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:14.092000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.092496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.092974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.119051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:14.120643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:14.135925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 180/344] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:14.135925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5d7b-f35f-4a9c-b40d-313343ef7656 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.135925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5d7b-f35f-4a9c-b40d-313343ef7656 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.135925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5d7b-f35f-4a9c-b40d-313343ef7656 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.138933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 181/345] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:14.139768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa5d7b-f35f-4a9c-b40d-313343ef7656 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.211049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 165/346] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:14.218717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.259654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa5d7b-f35f-4a9c-b40d-313343ef7656 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.288529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.297252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.297985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.298266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.298538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.359410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.371044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe817-d3d7-45b9-b534-721ce64f0321 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.393544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.395786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.435262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.436325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.436620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.438245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.439922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.442278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141dfcc-de33-4a5a-9dc4-79f9eb004115 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.445330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 166/347] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:14.447419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.447715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.448272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.471968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:14.473188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:14.545481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 182/348] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:14.546161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.557607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 167/349] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:14.558743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.580387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.663694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836f70-f989-4bf0-b2df-0802faba596a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.677986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.701745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e6afe-2b30-43b5-b29c-0ac4329f9235 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.710832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.711867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.712305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.712743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.725466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-9e8e3e87-6d08-4314-b1c1-5161ab586939 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:14.733992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:14.734915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 19:25:14.735851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:14.736474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 183/350] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:14.737609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8556a-10d8-432d-9bff-7ebaa9f8a02a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.737900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8556a-10d8-432d-9bff-7ebaa9f8a02a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.738184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8556a-10d8-432d-9bff-7ebaa9f8a02a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.739621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 184/351] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:14.740201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8556a-10d8-432d-9bff-7ebaa9f8a02a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.756187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.757350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.757633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.758398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.766483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.767979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.769966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 168/352] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:14.773029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.853343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:14.875432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8556a-10d8-432d-9bff-7ebaa9f8a02a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.892529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060d39-f916-48d2-95c7-ca4dc71fee37 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:14.918640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.919556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.919838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.920127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.921278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:14.922705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42663d93-cbc3-4d4d-90a7-019679936c2c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:14.924844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 185/353] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:14.925910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da04ec-4701-4498-ae32-984d52343181 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.926164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da04ec-4701-4498-ae32-984d52343181 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.926438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da04ec-4701-4498-ae32-984d52343181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.943790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:14.944820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:14.945098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:14.945381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:14.951409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-62da04ec-4701-4498-ae32-984d52343181 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:14.952609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-62da04ec-4701-4498-ae32-984d52343181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:14.959525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:14.969623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:14.970559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:14.971927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 169/354] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:14.972588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:14.989077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.037218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 186/355] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:15.038222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da04ec-4701-4498-ae32-984d52343181 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.095467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004423e1-34c8-4038-98db-58d81797ba1f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.136998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.138698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.139218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.139713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.140770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:15.141273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.142978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.148746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.154263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 170/356] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.155530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.170797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da04ec-4701-4498-ae32-984d52343181 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.214026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.214666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.214972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.215309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.216756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=9e8c266acd9344289570f2dc0855e7ca) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:15.217072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.218620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.225151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 187/357] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.225831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.245900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.290017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41af66-9da4-491b-9940-99ee0cfec012 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.332849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.333732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.334012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.334288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.335531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.337005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.338707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 171/358] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.340163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.375414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f5500-f186-45dc-9b58-4a085fe0e7ec None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.426247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.427223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.427709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.428193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.429627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:15.430166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.433811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.461516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 188/359] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.461516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.493110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.498148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d42c-89fd-4ea6-814a-461ab1460449 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.539962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.539962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.540252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.540494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.554148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.559262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-bea92c63-ba7b-40b8-a58e-f09ab8184da9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:15.567934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:15.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 172/360] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:15.573734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e20e-8342-459d-87d8-2a58f89a8b20 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.574442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e20e-8342-459d-87d8-2a58f89a8b20 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.574442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e20e-8342-459d-87d8-2a58f89a8b20 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.574442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 173/361] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:15.574442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e20e-8342-459d-87d8-2a58f89a8b20 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.617220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21aa9-22ba-477c-9987-601111db1140 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.659645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.660826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.661118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.661401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] PATH_INFO: `/v3/users/c91184697c3e47de94b21656b1cb97db` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.664463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] RBAC: Authorizing `identity:get_user(user_id=c91184697c3e47de94b21656b1cb97db)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.665285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.668273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 189/362] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/users/c91184697c3e47de94b21656b1cb97db => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.669648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.703369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e20e-8342-459d-87d8-2a58f89a8b20 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.759876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.760983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.761498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.761981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.763298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=9e8c266acd9344289570f2dc0855e7ca) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:15.763931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.765935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.773285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 174/363] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.773285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:15.807459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.830516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b6858-a2e8-4d66-9b01-f61869a8c604 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.863826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:15.869689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.870357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.870643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.870934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.909242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978b04b-ae0c-402f-a6b4-93329edbd1ab None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:15.949279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.949443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:15.949790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.949889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.950073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.950958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.952657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:15.952657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:15.954202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2228c39-f82f-45fe-8873-0b841d583b89 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:15.962175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 175/364] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:15.963472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:15.963759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:15.964036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:15.983314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:15.984497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:16.056115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.061647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 176/365] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:16.062377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.081651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 190/366] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:16.083421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.124388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.184293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f00f2e-25e4-4330-9e53-2e96569bdf51 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.199713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3529-e2bb-4bf5-b9c8-3b888a2606f0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.235008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.235966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.236450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.236968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.238079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] RBAC: Adding query filter params (role.id=7e44f86ffd224a0bac4489c48e9da64e, user.id=c91184697c3e47de94b21656b1cb97db, scope.system=all) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:16.238571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.240529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.242300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.243036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.243324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.243603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.254311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 177/367] 10.222.0.36 () {56 vars in 1308 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/role_assignments?user.id=c91184697c3e47de94b21656b1cb97db&scope.system=all&role.id=7e44f86ffd224a0bac4489c48e9da64e => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.255448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.300095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.323924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.325673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.366790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6b7f-2122-4edc-bea1-71908481fc1e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.380881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.403059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.404838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.405304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.405753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.407218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.408861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.411203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 178/368] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.412244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.445763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 191/369] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:16.447690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.523493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211467-3b1e-4810-aaff-bc499d858e54 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.551289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.559937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.560806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.561165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.561515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.564813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b370-2735-4e54-83a7-07c0a2d55ef7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.573927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:16.583254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:16.584195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:16.585001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 179/370] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:16.585738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.605146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.606274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.606732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.607439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.608680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.612078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.613394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 192/371] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.614251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.638999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.716938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943d50-a44f-46bf-af1e-604979c69a60 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.750197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28dc94-3450-40d0-a94b-31e183c2a937 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.766504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.767584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.767966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.768344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.772741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.774376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.776277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 180/372] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.777140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.790887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.791940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.792229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.792567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.793684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:16.794042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.795774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.808402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 193/373] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.809556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.826495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.918036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c2922-7e04-4503-8f67-b760822e93ce None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.929082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3b078-5b18-47b9-9837-9d7f63964fc0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:16.929479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:16.962586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.963276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.963833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.964125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.965135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=1d142bb142314799b8554ac8d6275c5d) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:16.965452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.967540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.970465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:16.971446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:16.971801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:16.972151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:16.973387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:16.973767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 181/374] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.974569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:16.974823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:16.981131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 194/375] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:16.988170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:17.115982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc955db-71e1-4395-bd23-2d2bddcdc18c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:17.160440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:17.161126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:17.161408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:17.161691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:17.195522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:17.222893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ca31-18c5-4755-9b56-b7cb3691cb04 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:17.223611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:17.279025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:17.280105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:17.280978 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:17.282667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:17.283198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:17.283676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:17.284717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=1d142bb142314799b8554ac8d6275c5d) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:17.287070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:17.288795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:17.302025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 195/376] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 260 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:17.427010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 182/377] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:17.437406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:17.577489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4689a4-e061-4b67-903e-783bde40e1e7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:17.615444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:17.616667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:17.617149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:17.617623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] PATH_INFO: `/v3/projects/adf859e3c6024ed7b109d2f0ca77139a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:17.619083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] RBAC: Authorizing `identity:get_project(project_id=adf859e3c6024ed7b109d2f0ca77139a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:17.620800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:17.623372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 196/378] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:17 2022] GET /identity/v3/projects/adf859e3c6024ed7b109d2f0ca77139a => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:17.628803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:17.720695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:17.754656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3efae-3162-4513-a7f0-5398b9504958 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:17.802452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:17.803356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:17.803646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:17.803932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] PATH_INFO: `/v3/groups/5ef1b44df7654194be63a9ef2caa5476` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:17.811568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] RBAC: Authorizing `identity:get_group(group_id=5ef1b44df7654194be63a9ef2caa5476)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:17.813312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9313f-ff77-4b5c-8828-bfc957986e45 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:17.815387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 183/379] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:17 2022] GET /identity/v3/groups/5ef1b44df7654194be63a9ef2caa5476 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:17.821040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:17.846753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:17.974687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3cd0-3489-4aea-a4c5-33a3e8db82ff None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:18.015288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:18.015962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.016247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.016525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.017374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] RBAC: Adding query filter params (group.id=5ef1b44df7654194be63a9ef2caa5476, role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=adf859e3c6024ed7b109d2f0ca77139a) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:18.017728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:18.019269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a2b0-a606-4c78-a6f5-56ad14119696 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:18.025860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 197/380] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:17 2022] GET /identity/v3/role_assignments?group.id=5ef1b44df7654194be63a9ef2caa5476&scope.project.id=adf859e3c6024ed7b109d2f0ca77139a&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:18.119506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e293c-f69c-4e0b-96df-9ca54165252d None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.120064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e293c-f69c-4e0b-96df-9ca54165252d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.120064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e293c-f69c-4e0b-96df-9ca54165252d None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.121951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 184/381] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:18.128383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.128671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.128955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.152090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:18.153363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:18.269926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 198/382] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:18.314351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e293c-f69c-4e0b-96df-9ca54165252d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:18.448670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e293c-f69c-4e0b-96df-9ca54165252d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:18.492641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:18.493446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.493729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.494007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.504141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:18.515826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:18.540499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:18.541512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:18.542371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 185/383] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:18.543140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:18.543140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:18.667488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bda96e-8da5-41c3-8d3e-8f5720dba967 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:18.706565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:18.707550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.707996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.708431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.709347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:18.709807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:18.711358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaba990-b008-454b-b1e4-67826af60271 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:18.722324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 199/384] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:18.727483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:18.749720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a6f50-acbc-493d-a45a-3d940039f2ec None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.750254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a6f50-acbc-493d-a45a-3d940039f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.750716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a6f50-acbc-493d-a45a-3d940039f2ec None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.755429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 200/385] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:18.757386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:18.757916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:18.758403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:18.776642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:18.778688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:18.892056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:18.895198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 201/386] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:18.924365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313d60a-8967-4e3c-93f0-8c30e98aa647 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:18.934325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.005518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.006784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.007316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.007814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.009991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.011760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.017242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 186/387] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:18 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:19.021447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.145268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580430a-ddd2-4ab4-80e7-a4ab393c0f5a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.147127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:19.202434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546f2d9-3a4f-416e-bbce-4be81dcc9286 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.249512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.250473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.250720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.251033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.254094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.255247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.255841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.256330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.261689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:19.263200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.264860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.267127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 187/388] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:19 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:19.270448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:19.271754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:19.272568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 202/389] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:19.274284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.277117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.338756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:19.386250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0b251-e086-4974-9390-193915145f15 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.416692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70196ba8-e5ac-4ba2-9658-270c1e96aa37 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.423393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.424362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.424847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.425325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.426278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:19.426789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.428379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.438000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 203/390] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:19.443300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.478821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:19.490539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.491492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.493216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.493216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.494109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, group_id=2897f89bdcc149f9be25d3a9bbc06d92, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.495621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.573731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee58c-2654-47d8-afea-4052e3908415 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.593491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:19.614274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.615513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.615657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 188/391] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Jun 16 19:25:19 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:19.616130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.616650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.626271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.631160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8c706-2575-49c7-b991-06e4228dd997 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.631160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 204/392] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:19 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:19.636203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.686200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b428-9fce-4f4f-beab-a2737d26d4d5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.686828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b428-9fce-4f4f-beab-a2737d26d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.687602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b428-9fce-4f4f-beab-a2737d26d4d5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.689485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 205/393] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:19.695345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.695922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.696407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.714062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:19.715767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:19.790345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 206/394] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:19.814495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.853089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0383-ad7d-4b7b-9819-080f6809d318 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.902041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:19.903351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:19.904286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:19.904800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:19.906684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:19.908921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:19.911926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 189/395] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:19 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:19.916947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:19.941482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a58c5c-ad34-4dd2-8cb2-60c53783a3f9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:19.964760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.000190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.001549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.002704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.003402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.006222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.024883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:20.042040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:20.043115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:20.044569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:20.048183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 207/396] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:20.054990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.085963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66e01-0020-48e7-9cd2-e9ea15578811 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.152389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.154588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.155123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.155620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.166141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c266acd9344289570f2dc0855e7ca, group_id=2897f89bdcc149f9be25d3a9bbc06d92, role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:20.167979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:20.230566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db51bc0-8e52-445f-963a-e8510b72c9fc None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.266250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.267450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.267956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.268464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.269518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] RBAC: Adding query filter params (name=member) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:20.270088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:20.271841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8a0a-88a4-48c6-aaa2-6c8c2221d46d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:20.286802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 190/397] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Jun 16 19:25:19 2022] PUT /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:25:20.295505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 208/398] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:20.298138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.360682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9868f6-34fc-4618-9a06-94210f04a98c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.360982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9868f6-34fc-4618-9a06-94210f04a98c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.361265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9868f6-34fc-4618-9a06-94210f04a98c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.362897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 209/399] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:20.370798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.371140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.371424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.400452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:20.401664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:20.432266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efac4-6c2b-49b2-b2e3-1c83cd50f91a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.457004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.474810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.476263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.476696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.477118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.478636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:20.479581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.482141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:20.484383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 191/400] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:20.490893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.553028 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 210/401] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:20.588098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.623026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ec32-6103-4eea-8424-51001dfbe12f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.674306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.675727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.676157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.676843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.679134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:20.680740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121385de-c235-4f7c-aaad-9f720f46c315 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:20.683443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 192/402] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:20.690181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5af3a-eb24-4e68-a85d-bd0ab1e795b2 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.690181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5af3a-eb24-4e68-a85d-bd0ab1e795b2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.690181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5af3a-eb24-4e68-a85d-bd0ab1e795b2 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.691001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 193/403] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:20.691861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5af3a-eb24-4e68-a85d-bd0ab1e795b2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.720791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f4e2-cf58-4028-8158-6da73b81e7ec None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.760174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.761098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.761484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.761882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.767757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.775386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-3992f797-7acc-440d-9263-453c35e6074b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:20.784010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:20.784906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:20.785744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:20.788426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 211/404] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:20.794797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec17574-2219-4933-9f83-375213f6c51d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.794797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec17574-2219-4933-9f83-375213f6c51d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.794797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec17574-2219-4933-9f83-375213f6c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.809254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cec17574-2219-4933-9f83-375213f6c51d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:20.812460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cec17574-2219-4933-9f83-375213f6c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:20.817800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5af3a-eb24-4e68-a85d-bd0ab1e795b2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:20.839923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:20.861267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:20.862831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:20.863408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:20.863841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:20.866694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, group_id=2897f89bdcc149f9be25d3a9bbc06d92, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:20.868470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:20.888991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 212/405] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:20.890215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec17574-2219-4933-9f83-375213f6c51d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:20.964829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 194/406] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Jun 16 19:25:20 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:20.965920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.035983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec17574-2219-4933-9f83-375213f6c51d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.083690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.084624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.085054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.085502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.086429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:21.086997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.089841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.101103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 213/407] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:21.107630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.135318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:21.237158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a09c-4478-4ce0-bd6b-36f2cd72c73e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.238708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4884-d54a-4b5e-980f-76ec7b6c6016 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.257832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:21.277697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.278615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.278901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.279263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.283572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.285236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.285871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.286356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.288154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.289882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.290686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:21.292301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 214/408] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:21 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:21.298266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:21.300552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:21.301633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:21.302583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 195/409] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:21.311957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.403470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:21.434969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5ea8d-624d-43e8-8707-08951fc61407 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.487439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f8d6-d834-4be5-a03c-9197cc8b1b1e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.505155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.506303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.506594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.506877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.511812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.513406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.518226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 215/410] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:21 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:21.521505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.545916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.546543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.546825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.547128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.547912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:21.548222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.549641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.571147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 196/411] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:21.577074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.655199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:21.668003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129db81-28d6-4939-9dc9-28a10d91759d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.707229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.708614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.709856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.710955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.716305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=1d142bb142314799b8554ac8d6275c5d, group_id=2897f89bdcc149f9be25d3a9bbc06d92, role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.719076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.758394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87e077-d852-4e9a-98c0-13dd757d9122 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:21.788297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 216/412] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Jun 16 19:25:21 2022] PUT /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d/groups/2897f89bdcc149f9be25d3a9bbc06d92/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:21.807890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:21.809603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:21.810212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:21.810699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] PATH_INFO: `/v3/groups/5ef1b44df7654194be63a9ef2caa5476` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:21.812710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] RBAC: Authorizing `identity:get_group(group_id=5ef1b44df7654194be63a9ef2caa5476)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:21.814448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:21.819137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 197/413] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:21 2022] GET /identity/v3/groups/5ef1b44df7654194be63a9ef2caa5476 => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:21.824442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:21.847514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:21.951101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba2c8a-0913-41de-85fe-2523862b0c0f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:22.001460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:22.002346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.002631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.002908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] PATH_INFO: `/v3/projects/adf859e3c6024ed7b109d2f0ca77139a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.004185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] RBAC: Authorizing `identity:get_project(project_id=adf859e3c6024ed7b109d2f0ca77139a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:22.005665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d3179-984a-46b2-8301-c004d29fb2ad None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:22.012107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 217/414] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:21 2022] GET /identity/v3/projects/adf859e3c6024ed7b109d2f0ca77139a => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:22.014374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:22.125974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea89dc-2390-4240-a014-68c9b0d8c78a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:22.133504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:22.170958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:22.170958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.170958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.170958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] PATH_INFO: `/v3/projects/adf859e3c6024ed7b109d2f0ca77139a/groups/5ef1b44df7654194be63a9ef2caa5476/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.171272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] RBAC: Authorizing `identity:create_grant(project_id=adf859e3c6024ed7b109d2f0ca77139a, group_id=5ef1b44df7654194be63a9ef2caa5476, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:22.172870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8b849-cfda-4609-a1ed-991b428e2b9e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:22.251982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 198/415] 10.222.0.36 () {58 vars in 1442 bytes} [Thu Jun 16 19:25:22 2022] PUT /identity/v3/projects/adf859e3c6024ed7b109d2f0ca77139a/groups/5ef1b44df7654194be63a9ef2caa5476/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:22.354428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112974-a723-48a9-b8cd-62a0ec9185af None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.354974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112974-a723-48a9-b8cd-62a0ec9185af None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.355054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112974-a723-48a9-b8cd-62a0ec9185af None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.356752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 218/416] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:22.362197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.362806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.363308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.390351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:22.391975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:22.492611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 199/417] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:22.516772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8112974-a723-48a9-b8cd-62a0ec9185af None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:22.621306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:22.639540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8112974-a723-48a9-b8cd-62a0ec9185af None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:22.677139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:22.677869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.678155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.678434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.689934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-b270107a-37bf-497f-9d69-e46c95aed74c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:22.697745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:22.698627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:22.699467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 219/418] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:22.706329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:22.801176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1befa2-419e-4f71-aa15-6de8e009ef29 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.801515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1befa2-419e-4f71-aa15-6de8e009ef29 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.801737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1befa2-419e-4f71-aa15-6de8e009ef29 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.803167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 220/419] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:22.808273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.808567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.808853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.812475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:22.818363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25ca50-a513-4a70-aac8-e4dc97ec0975 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:22.847975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:22.849188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:22.853153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:22.853823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:22.854109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:22.854390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:22.855178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:22.855502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:22.857409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:22.866608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 200/420] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:22.871605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:22.978796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbeba0-e180-43e4-ada3-3fdd8d7ba0d1 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:22.983330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 221/421] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:23.012313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.014691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.018963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.018963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.018963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.075089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.134892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.143014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.144475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.192275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690f124-5c4f-408c-849d-0569c525d022 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.260348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.261088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.261369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.261648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.291887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:23.306949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:23.307862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:23.308676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:23.311110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 222/422] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:23.318406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 201/423] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:23.319877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.323655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.439796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.460801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44585ec9-b197-4e46-93fc-6636c469f6f2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.511947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d168aa-9204-42e3-b1e5-6a6589c656ad None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.546894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.548046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.548338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.548614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.549806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.551266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.553447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 202/424] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:23.558602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.574557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.576595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.577305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.577793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.579431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:23.579929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.581526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d41a-4d6f-451f-8dac-faa0bda12199 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.594730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 223/425] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:23.595246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.599501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb0f6-4fb4-4e80-b730-a902e868f597 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.600233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb0f6-4fb4-4e80-b730-a902e868f597 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.601633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb0f6-4fb4-4e80-b730-a902e868f597 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.603153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 224/426] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:23.607827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb0f6-4fb4-4e80-b730-a902e868f597 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.682784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f669-414f-4860-875d-34aa36fb0124 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.709549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.718390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.718695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb0f6-4fb4-4e80-b730-a902e868f597 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.719417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.719773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.720121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.721437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.722954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecb76e-cd98-4bd7-939d-99c2a937e6ea None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.724629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 203/427] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:23.730567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.730949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.731320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.747913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:23.749233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:23.757771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.758791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.759312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.759742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.827032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 204/428] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:23.827032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.843542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.851676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.853387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.925310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e82a0-e50f-49a5-af37-6670132a9926 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:23.950679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:23.962419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:23.963362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:23.963806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:23.964241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:23.965256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=9e8c266acd9344289570f2dc0855e7ca) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:23.965880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:23.967563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:23.973883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 205/429] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:23.974877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:23.986977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 225/430] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:23.988305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.093396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4af5-778d-4edd-a205-c95ed0799dd1 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.131664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.132840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.133341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.133810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.146426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:24.154942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:24.155919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 19:25:24.156703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:24.159681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 206/431] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:24.167308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.209847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.216312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3989ebb-09c4-474c-82ee-56bb000af3ce None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.228654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.264025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.264903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.265188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.265470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] PATH_INFO: `/v3/projects/9e8c266acd9344289570f2dc0855e7ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.266675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8c266acd9344289570f2dc0855e7ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.268277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.274507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 226/432] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:23 2022] GET /identity/v3/projects/9e8c266acd9344289570f2dc0855e7ca => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:24.275849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.286119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3785a9-dbd3-4577-9532-e76138230029 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.324699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.326111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.326766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.327245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.328273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:24.328757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.330394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.341298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 207/433] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:24.353504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.391594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcc86-18a5-4ff2-9201-b308ee35ce35 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.429056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.429928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.435027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.435027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.435027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.435027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.439102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 227/434] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:24.442380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.467548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c15ed9-46e2-4c1f-87b1-cbff10798fc3 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.477974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.483361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.527760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.529242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.529534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.529817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.592576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4feca9-ba19-4839-945d-884a322cd094 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.622173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.623742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.633569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.634243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.634529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.634958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.635796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=9e8c266acd9344289570f2dc0855e7ca) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:24.636113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.637696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf17807-c61b-459e-ba90-46a4b5a24e20 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.644941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 228/435] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=9e8c266acd9344289570f2dc0855e7ca&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:24.652690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eba9-1431-472f-9498-14e8b3a3c4c2 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.653546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eba9-1431-472f-9498-14e8b3a3c4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.653807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eba9-1431-472f-9498-14e8b3a3c4c2 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.655637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 229/436] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:24.661388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.661677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.662068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.689517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:24.690764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:24.752354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 208/437] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:24.761125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.774625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.781729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 230/438] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:24.809693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.857081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:24.892241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67b39-0863-4992-90c2-c4a8c12e9bf7 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:24.932046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:24.932905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.933189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.933465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.934806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:24.936330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561e3e-26e3-46a3-b461-c644fd0bd39e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:24.939003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 209/439] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:24.940644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf28c-67ee-4e35-ac01-07050d99b78b None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:24.941138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf28c-67ee-4e35-ac01-07050d99b78b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:24.941602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf28c-67ee-4e35-ac01-07050d99b78b None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:24.943578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 210/440] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:24.948129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cf28c-67ee-4e35-ac01-07050d99b78b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:24.952630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a913-e8ba-4e38-987d-223c8fc1fbcf None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.000935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.001798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.002082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.002366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.014610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-d7e861b6-9cff-42c5-82e2-58230574ce20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:25.023640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:25.024576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:25.025373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 231/441] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:25.026046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.026420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.026842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.051789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:25.052976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:25.053163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.077171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cf28c-67ee-4e35-ac01-07050d99b78b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.117680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.118532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.118816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.119132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.120295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.121740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.123407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 211/442] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:24 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:25.124241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.130612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 232/443] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:25.131485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.150224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.226671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f6d76-88a3-4e90-b6c4-af337df95848 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.231144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fa57-6462-41e7-b578-4ad51852786f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.259526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.260189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.260477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.260762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.261515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:25.261832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.263798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.265480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.266128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.266407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.266681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.267537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=c67c387f18884b4081e6a7754b8319a6, scope.project.id=1d142bb142314799b8554ac8d6275c5d) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:25.267845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.269248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.271589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 212/444] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:25.272371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.275092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 233/445] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=c67c387f18884b4081e6a7754b8319a6 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:25.277013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.285468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.369404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.390176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe26b3-bc85-4ea6-9f65-e7744208e8a8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.391133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6017cb-d9d1-448f-8080-3a0e1d82f7b0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.429308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.430027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.430319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.430691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.431960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.432616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.432904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.433178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.443392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:25.451785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:25.452814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:25.453911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 213/446] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:25.458688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.517334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.518804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.554246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.594366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc57c9f-8acd-4e8a-8e58-3610c8b9b373 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.611848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.635829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 234/447] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:25.636993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.637707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.637991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.638268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.639086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:25.639399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.640638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.640776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.650514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 214/448] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:25.656118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.738554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556c44-1e2f-4363-adff-57c5c9c22dcd None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.754051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bb31-a93e-4856-bfc2-45b0b523e6f3 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.771594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.772456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.772746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.773023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] PATH_INFO: `/v3/projects/1d142bb142314799b8554ac8d6275c5d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.774206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] RBAC: Authorizing `identity:get_project(project_id=1d142bb142314799b8554ac8d6275c5d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.775674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.777352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 235/449] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/projects/1d142bb142314799b8554ac8d6275c5d => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:25.782032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.782858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.787683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.788329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.788615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.788942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.832944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:25.855600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.857587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.882948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2f5a-69fc-4800-82e7-6854a1e8b073 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:25.920195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:25.921068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:25.921350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:25.921625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] PATH_INFO: `/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:25.922808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] RBAC: Authorizing `identity:get_group(group_id=2897f89bdcc149f9be25d3a9bbc06d92)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:25.924416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:25.926141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 236/450] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/groups/2897f89bdcc149f9be25d3a9bbc06d92 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:25.932018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.959861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 215/451] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:25.965905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:25.994479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:26.030410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf2368-ea30-46bd-812f-f89905d7544c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:26.046469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:26.061616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42647f9-b248-446d-aa63-ca12f28d5ee2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:26.070079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:26.070760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:26.071076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:26.071357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:26.072195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] RBAC: Adding query filter params (group.id=2897f89bdcc149f9be25d3a9bbc06d92, role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=1d142bb142314799b8554ac8d6275c5d) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:26.072505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:26.073943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:26.079869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 237/452] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/role_assignments?group.id=2897f89bdcc149f9be25d3a9bbc06d92&scope.project.id=1d142bb142314799b8554ac8d6275c5d&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:26.100422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:26.101309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:26.101596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:26.101876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] PATH_INFO: `/v3/projects/adf859e3c6024ed7b109d2f0ca77139a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:26.103166 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] RBAC: Authorizing `identity:get_project(project_id=adf859e3c6024ed7b109d2f0ca77139a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:26.104664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:26.107030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 216/453] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:25:25 2022] GET /identity/v3/projects/adf859e3c6024ed7b109d2f0ca77139a => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:26.112820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:26.219305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321f17-c45f-46e0-a1ab-d91322f3c2a7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:26.253229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:26.254020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:26.254301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:26.254575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] PATH_INFO: `/v3/groups/5ef1b44df7654194be63a9ef2caa5476` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:26.255816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] RBAC: Authorizing `identity:get_group(group_id=5ef1b44df7654194be63a9ef2caa5476)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:26.257269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940527d-9cdd-464c-8309-376f03caf6a2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:26.259023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 238/454] 10.222.0.36 () {56 vars in 1180 bytes} [Thu Jun 16 19:25:26 2022] GET /identity/v3/groups/5ef1b44df7654194be63a9ef2caa5476 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:26.265298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:26.365352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:26.369174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ff659-a137-445c-9d62-c45bfa240298 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:26.403868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:26.404540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:26.404831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:26.405110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:26.405962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] RBAC: Adding query filter params (group.id=5ef1b44df7654194be63a9ef2caa5476, role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=adf859e3c6024ed7b109d2f0ca77139a) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:26.406280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:26.407765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2821a7-c05c-4677-93d5-77da947618d2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:26.413736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 217/455] 10.222.0.36 () {56 vars in 1376 bytes} [Thu Jun 16 19:25:26 2022] GET /identity/v3/role_assignments?group.id=5ef1b44df7654194be63a9ef2caa5476&scope.project.id=adf859e3c6024ed7b109d2f0ca77139a&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:28.914450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae43d3-1b24-43f4-9dd1-59e4d597ccd4 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:28.914992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae43d3-1b24-43f4-9dd1-59e4d597ccd4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:28.915107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae43d3-1b24-43f4-9dd1-59e4d597ccd4 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:28.916682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 239/456] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:28.922242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603003-5edd-461f-9408-d1dbe9b32746 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:28.923118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603003-5edd-461f-9408-d1dbe9b32746 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:28.923597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c603003-5edd-461f-9408-d1dbe9b32746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:28.951827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5c603003-5edd-461f-9408-d1dbe9b32746 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:28.953302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5c603003-5edd-461f-9408-d1dbe9b32746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:29.041931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 218/457] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:29.080245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae43d3-1b24-43f4-9dd1-59e4d597ccd4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.144796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:29.223394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae43d3-1b24-43f4-9dd1-59e4d597ccd4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:29.278736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:29.279390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.279676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.279956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.280306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d65a-588c-4a81-ad65-556374063a96 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.280696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d65a-588c-4a81-ad65-556374063a96 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.281205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d65a-588c-4a81-ad65-556374063a96 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.283077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 219/458] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:29.287708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597796c-cfc1-4bda-9f90-50aa6dc9db54 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.288000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597796c-cfc1-4bda-9f90-50aa6dc9db54 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.288282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597796c-cfc1-4bda-9f90-50aa6dc9db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.296503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:29.297958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:29.316300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:29.317226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:29.318227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:29.318227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:29.318227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.318227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:29.318227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 240/459] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:29.324005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e597796c-cfc1-4bda-9f90-50aa6dc9db54 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:29.325943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e597796c-cfc1-4bda-9f90-50aa6dc9db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:29.330827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.453975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 220/460] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:29.475503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed5cd-478d-41bd-9654-0ace520567a5 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.475996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed5cd-478d-41bd-9654-0ace520567a5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.476442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed5cd-478d-41bd-9654-0ace520567a5 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.477938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 221/461] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:29.482656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.483117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.483537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.498985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:29.501830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b55ed-35b1-422f-96a5-db58225db071 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:29.503537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:29.505485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:29.526447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:29.536536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:29.537709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.538174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.538629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.539579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:29.539885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:29.541490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:29.554167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 241/462] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:29.555283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.599078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 222/463] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:29.600460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.685093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8276-d0d2-4ccf-8598-ae9fe94e97f3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:29.729825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:29.731194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.731957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.732715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.740706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c77bf-4684-4219-a779-0d94811d39c4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:29.746020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:29.748214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:29.770460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:29.771653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:29.772624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:29.772624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:29.772624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.772624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:29.774894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 242/464] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:29.776145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.792763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:29.793812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:29.794652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.795036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.795743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.796164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:29.797715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:29.882824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e0552-7584-42c1-a134-1afd946e9024 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:29.916343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 223/465] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:29 2022] POST /identity/v3/users => generated 246 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:29.917411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:29.929999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:29.946487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:29.948712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:29.949222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:29.950464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:29.966116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:29.968799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-7b24d9b4-71b1-4e53-b6c5-5ea469cbd238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:29.994690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:29.995784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:29.997534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:29.997534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:29.997534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:29.997534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:29.999449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 243/466] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:30.004793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54ff1-b13c-4df1-9337-d652b5cba9e4 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.004904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54ff1-b13c-4df1-9337-d652b5cba9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.005142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f54ff1-b13c-4df1-9337-d652b5cba9e4 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.006729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 244/467] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:30.011436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f54ff1-b13c-4df1-9337-d652b5cba9e4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.050334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc858c-6151-4555-9b90-86d74a3e001f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.090721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.091085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.091374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.091660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.092451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:30.093740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.095206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de76d10-8677-4900-b778-9a8a22154392 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.103103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.116209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 224/468] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:30.118978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.118978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.118978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.137803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:30.139365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:30.158335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f54ff1-b13c-4df1-9337-d652b5cba9e4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.210085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.210760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.211181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.211465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.212199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:30.212511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.213899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.225137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.231613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 225/469] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:30.232911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.243646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 245/470] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:30.251132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.353691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026af15-72a1-4671-a7cf-a5e74b5aa20e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.370651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d2d64-bff0-4eff-a1e2-177ba9d73432 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.390123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.391180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.391705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.392051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.392758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.394289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.395815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.406645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.407346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.419420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.419420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.419420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.419420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46767a-69a3-4fac-bd58-d86cdf4b73a8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.468694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 226/471] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:30.469525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.492898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 246/472] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/users => generated 248 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:30.494619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa74d3-ab6d-4c2b-b068-bbfc9d26893d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.494937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa74d3-ab6d-4c2b-b068-bbfc9d26893d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.495220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa74d3-ab6d-4c2b-b068-bbfc9d26893d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.501554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 247/473] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:30.503277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.503575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.503859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.523902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:30.525459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:30.541649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.624491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 248/474] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:30.650826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.667741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03cca4-6761-453f-b360-fb8a84b9abcd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.713872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.715862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.716260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.716541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.732641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.734272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:30.754467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:30.755423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:30.756293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:30.756293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:30.756293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:30.756293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:30.756293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 227/475] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:30.762690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.780793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612465-da0a-411b-8c4b-bfb01c5dfa2c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.794828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.825967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.827706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.829036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.829467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.830229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.831687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.878712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.sql.core [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 19:25:30.878712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 19:25:30.878712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [parameters: {'id': '35e78688214b4cdfbd159e4ad498217a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 19:25:30.878712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80308) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:25:30.896116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:25:30.897046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application yield resource Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise self.value Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application session.commit() Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 19:25:30.898392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.flush() Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application compat.raise_( Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 19:25:30.899468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:25:30.900347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:25:30.901161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [parameters: {'id': '35e78688214b4cdfbd159e4ad498217a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:30.903491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 19:25:30.905426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:30.907385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 249/476] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:30.907385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:30.914360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14f042-616a-4528-b9ac-d186e9aefa1d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:30.921556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:30.951901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:30.953379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:30.953892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:30.954365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:30.955399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:30.956001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:30.957589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:30.971006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 228/477] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:30.976815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:31.045554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4bb0-35ab-4ba8-b748-ae5a467745e2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:31.081908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0431e33-0739-40d7-88f7-7442aa548b4f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:31.109293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:31.110163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:31.110451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:31.110724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:31.118448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:31.119479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:31.119965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:31.120437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:31.121351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:31.123049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026395-565c-4eb2-9ce9-78e0573f5e10 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:31.125177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:31.139752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:31.143241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:31.144081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 250/478] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:25:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:31.146837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:31.222459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:31.231494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 229/479] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:30 2022] POST /identity/v3/users => generated 249 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:31.254359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff748db-09d5-41c7-9d9d-59f9b927f387 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:31.298184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:31.298795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:31.299116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:31.299506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:31.300173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:31.300483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:31.301879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c152ee-d3cd-426f-a2af-c3d0d950e468 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:31.311664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 251/480] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:32.522296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f273b-9e6d-4483-aeb2-237267fa0751 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:32.522819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f273b-9e6d-4483-aeb2-237267fa0751 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:32.522948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f273b-9e6d-4483-aeb2-237267fa0751 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:32.524606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 230/481] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:32.530457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:32.530867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:32.531077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:32.552820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:32.554019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:32.650223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 252/482] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:32.696475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f273b-9e6d-4483-aeb2-237267fa0751 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:32.729701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:32.836622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f273b-9e6d-4483-aeb2-237267fa0751 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:32.880491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:32.881290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:32.881569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:32.881841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:32.896942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:32.905244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:32.906169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:32.907012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 231/483] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:32.912779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:32.951698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.051314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3a052-9855-43a4-bca0-73caf37580fd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.100605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.101216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.101494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.101764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.102518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:33.102827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:33.104220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:33.117471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 253/484] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:33.122550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.263043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.287099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cfe41-2b63-48a1-a52d-24f43c9642da None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.336640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.337841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.339164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.339443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.352053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:33.353486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d2bd6f4c-528b-4736-a33e-174d407240ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:33.381431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:33.382376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:33.383477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:33.383477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:33.383477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:33.383477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:33.383477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 232/485] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:33.389127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.425886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff670-e9cc-4363-b8a4-553fe1b0b329 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.426216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff670-e9cc-4363-b8a4-553fe1b0b329 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.426456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff670-e9cc-4363-b8a4-553fe1b0b329 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.428118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 233/486] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:33.432918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93737-b725-466d-9206-4fb4dc7c1bfe None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.439720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93737-b725-466d-9206-4fb4dc7c1bfe None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.439720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93737-b725-466d-9206-4fb4dc7c1bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.460392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7dc93737-b725-466d-9206-4fb4dc7c1bfe None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:33.461820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7dc93737-b725-466d-9206-4fb4dc7c1bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:33.493132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.505562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b05bd-49dd-49d7-877b-32a677e27827 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.543302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.544656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.545085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.545503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.546367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:33.546540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 234/487] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:33.547344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:33.548993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:33.551939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5328-37f6-4f0d-985a-862ce7876384 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.552377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5328-37f6-4f0d-985a-862ce7876384 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.552795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5328-37f6-4f0d-985a-862ce7876384 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.559417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 235/488] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:33.561061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.561345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.561622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.577363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 254/489] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:33.578289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.592940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:33.595189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:33.683408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.700239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 236/490] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:33.701530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.704190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b167e7-6b9e-460c-94f0-a06e870a1e49 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.741697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.742775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.743236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.743652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.755596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:33.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:33.765253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:33.766078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:33.768418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 255/491] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:33.769579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.781164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.824037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06f423-c588-482e-b533-994dc9d3bfe1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.858846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.859715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.860098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.860294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.867724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:33.869013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:33.887197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:33.888172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:33.889182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:33.893002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 237/492] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:33.893858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:33.899582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-f3cc-415b-ab76-01fb54364b56 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:33.944300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:33.945707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:33.946206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:33.946676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:33.955301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:33.961000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:33.970835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:33.973265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:33.974069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:33.978996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 256/493] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:33.980018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.005392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf3a5-d3ec-40fd-8ca3-ab2ed203a447 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.039443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.040151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.040433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.040704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.041459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:34.041762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.043766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.056926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 238/494] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:34.057839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.070829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.106419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43f37-616f-471a-b121-b51b985d946c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.146645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.147421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.147704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.147979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.148863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:34.149178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.150829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.194709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 257/495] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:25:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:34.195963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.198840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c49e-114c-4c95-b151-96c6c4988ed9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.217376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.234945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.235628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.235914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.236193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.236944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:34.237252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.238639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871eac6-29e9-43d4-a0e7-70bd27a6792b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.247060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 239/496] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:34.248274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88079f2-a18d-4b9a-ada9-614bfe48e1b3 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.248564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88079f2-a18d-4b9a-ada9-614bfe48e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.248840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88079f2-a18d-4b9a-ada9-614bfe48e1b3 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.254064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 240/497] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:34.254837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88079f2-a18d-4b9a-ada9-614bfe48e1b3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.297718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380b07b-901f-429a-a748-9b1030f5ba93 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.322836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.331378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.332422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.332712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.332994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.344312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.345794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.362104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88079f2-a18d-4b9a-ada9-614bfe48e1b3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-7d1dedb4-2e15-454c-bd7b-40fe5b85a995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:34.364751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:34.365651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:34.366490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:34.366490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:34.366490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.366490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:34.366490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 258/498] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:34.367190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838abb1-b4a6-478b-a8f6-cfeb8cc41d5d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.367461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838abb1-b4a6-478b-a8f6-cfeb8cc41d5d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.367737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838abb1-b4a6-478b-a8f6-cfeb8cc41d5d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.370980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 259/499] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:34.370980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838abb1-b4a6-478b-a8f6-cfeb8cc41d5d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.397764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.398412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.398696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.399007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.444290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.465888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.467375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.475049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838abb1-b4a6-478b-a8f6-cfeb8cc41d5d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.507949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.508954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.509237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.509514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.519767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.521189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-84642224-bcaa-40b4-bd1a-8b60913aa875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:34.538179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:34.539203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:34.540420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:34.540420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:34.540420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.540420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:34.540420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 260/500] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:34.540858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa2381-6824-40ea-95c9-a2e419c3f80a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.541141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa2381-6824-40ea-95c9-a2e419c3f80a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.541420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa2381-6824-40ea-95c9-a2e419c3f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.557537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-89aa2381-6824-40ea-95c9-a2e419c3f80a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:34.558732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-89aa2381-6824-40ea-95c9-a2e419c3f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:34.566967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 241/501] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:34.567792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.582589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.624795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 261/502] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:34.626078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.626374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.626645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.646312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.650285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:34.651500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:34.668321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042dea-60c7-407c-a360-d09cc1258f5a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.707955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.708618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.708897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.709175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.709895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:34.710201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.711609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.723112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 242/503] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:34.723845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.725775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 262/504] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:34.726589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.813452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.822576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6287d-54c3-4136-8a6d-c22e893920ba None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.825022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139dae81-4cfc-47a6-9b46-97c74274666d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.856138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.856787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.857070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.857345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.857546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:34.858055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:34.858337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:34.858425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.858622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:34.858897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:34.859754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.865871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:34.868783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:34.870233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:34.871274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 243/505] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:34.872092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:34.888698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:34.889805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:34.890671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:34.890671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:34.890671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:34.890671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:34.890671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 263/506] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:34.890970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:34.971252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e72b-ebca-48aa-b77a-ed629494c4a5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:34.989359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b33f1-41a5-435d-9de0-18f0afc3d2b4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.007498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.008294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.008584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.008874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.020238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:35.024812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.025640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.025922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.026199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:35.028203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.029111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 244/507] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.030637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.038522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.040392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:35.055975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.057094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 264/508] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.058016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.061221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.093833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.134741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d82ba-9cfd-457f-8dbe-8c16f54e0a23 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.158448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70be8f-3d13-4c2f-9912-6960b1e1c043 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.169122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.169910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.170195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.170470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.181101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.182547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.193688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.194482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.194771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.195086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:35.200639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.201648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.202517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:35.202517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:35.202517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:35.202517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:35.202517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 245/509] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.202779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.204057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.205502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:35.219464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.220498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 265/510] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.221409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.284002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.302415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91217-e9b2-462c-b7c7-4285d63a48c0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.313221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.321975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e88ca-8273-4e00-8c97-613c9dfbaae6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.336073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.337081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.337367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.337646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.338367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:35.338678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.340198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.351441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 246/511] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:35.352279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.363900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.364936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.365147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.365423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.366189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:35.366502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.368103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.377146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 266/512] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:35.377932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.462278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad257e3-6fa1-4bd5-9ee3-93e8b2db2a04 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.475084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf700bb-0ca6-4170-b85d-388b6d224c25 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.495300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.495968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.496301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.497093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.497939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:35.498259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.498394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.499751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.508326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.508986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.509277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.509559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.510282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:35.510593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.511994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.527778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 267/513] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:35.527778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.535971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 247/514] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:35.536657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.567067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.629657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eff13-c2d6-4280-b3e9-d70f0594c936 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.642314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff05e-0477-4eb3-8dd2-c551b3ac6fd5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.662959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.663635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.663928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.664201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.665039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:35.665352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.666773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.675097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.675876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.676159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.676436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.682845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.683831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 268/515] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:35.685212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.686100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:35.701197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.702291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 248/516] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.728674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.784943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beef67-f51f-4b46-8c81-f78fa5d99887 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.788524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.800826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52dd7c-0daf-451d-a61c-a7bdffce675d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:35.818916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.819733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.820021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.820300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.830006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.831475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.834449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:35.835139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:35.835423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:35.835696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:35.836389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:35.837806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:35.848990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:35.850016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:35.851077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:35.851077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:35.851077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:35.851077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:35.851077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 269/517] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:35.851523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.901317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 249/518] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:35 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:35.902348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:35.957633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:35.966629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af9bfb-fa88-429a-9b17-ae9cab1298ff None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.009560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.010495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.010831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.011343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.018889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc86b-12a7-43c9-a41b-262a4d57f5dd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.058415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.059083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.059379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.059664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.076653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.089120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.091239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.143523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.145008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.194815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.207398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 270/519] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:36.207398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.264858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 250/520] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:36.264858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.308874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.323462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50724a5d-cfdc-4ed6-bf28-89c9c3eaa700 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.359461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.360132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.360420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.360705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.361505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:36.361823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.363282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.369863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547a55f-cb01-4754-a61f-f7a59d91950d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.384863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 271/521] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:36.385573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.418154 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.418812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.419122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.419397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.420118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:36.420422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.421811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.437542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 251/522] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:36.442155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.455546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.504542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0975a9f-d907-461a-b27c-1da2a8721bf0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.539625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983e4f-f3d6-4d0a-9101-aa7dfbaaedba None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.541203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.541858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.542625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.542909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.543239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.553946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.555491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:36.573773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:36.575286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:36.576156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:36.576156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:36.576156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:36.576156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:36.576156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 272/523] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:36.576425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.581615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.582418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.582700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.583011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.593622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.595098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:36.613215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:36.614213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 252/524] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:36.615553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.688351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1ab1b-73e6-40f4-8fc2-6dcc5779b06b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.695768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.724599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76418-b394-407e-a6d5-aa0a4ef4cc9e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.725515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.726208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.726493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.726998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.727649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:36.727975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.729430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.735477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 273/525] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:36.736273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.772266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.773571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.773952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.774300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.783678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.785470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:36.802242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:36.803368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:36.804282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:36.808641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 253/526] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:36.810350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.836790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:36.880711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0945f-13ed-462d-a66c-4b76a9e3b3c0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.927521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:36.928223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:36.928478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:36.928771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:36.929492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:36.929802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:36.931314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:36.949663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 274/527] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:36.949663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:36.956195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b165f0d-6dc6-4698-aad5-379ac3d36a7d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:36.980310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.008062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.009694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.010464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.010993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.012714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:37.013387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.015436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.034546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 254/528] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:37.036587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.068750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a23-ede6-40c4-88f9-02fa3daee72a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.104832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.105424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.106273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.106549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.107430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:37.107738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.109410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.129424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 275/529] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Jun 16 19:25:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:37.130267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.141828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.178359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f557523-bf3d-4f23-90bd-0ed89c5a5ed2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.221530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.221759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.221969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.222239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.229255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.231602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.242061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:37.245662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:37.246535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:37.247416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:37.249798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 255/530] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:37.250773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.260587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bf00-b0a8-4e55-8c0c-6321bf55a9ac None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.307901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.308994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.309284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.309566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.318981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.320752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:37.344235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:37.345096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:37.345940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 276/531] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:37.346400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.355142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d960-b60a-47d3-a3c2-65730906899a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.390049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.391107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.391637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.392110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.396723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.464418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.467114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.481722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624fbd3-d01c-4fc7-8b6a-9734b5623b42 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.518173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.519259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.519937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.520679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.522230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:37.523027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.525315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.543798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 277/532] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:37.545014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.559694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.589456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 256/533] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:37.591659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.640798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.674867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5159f76-affc-4c1a-9cae-ba74975cb32a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.711592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.713016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.713495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.713986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.715551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:37.716202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.717834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.731467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 278/534] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:37.732178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2169c73-6799-4fce-8e61-b3f37712c570 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.732399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.784710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.785834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.786128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.786411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.787576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=56e334de6eee4d5eb8f2fafdb652e9b5) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:37.787900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.789618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:37.799068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 257/535] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/role_assignments?user.id=56e334de6eee4d5eb8f2fafdb652e9b5&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:37.800207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:37.826960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:37.895221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261de5-c0f5-4155-870b-de4270039b68 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:37.964219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:37.965488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:37.966067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:37.966546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:37.987580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:37.992451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.012531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:38.017459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b71e-50a1-45a1-a500-ff706c09f6bc None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:38.024483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:38.033092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 279/536] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:38.033935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.061733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.062941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.063242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.063539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.064397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=20bf88e1a4594d4f973c5dd049aeadd4) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:38.064719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:38.066219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eee2c-c4de-4bda-8dc8-15c5133ab6e7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.074047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 258/537] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:37 2022] GET /identity/v3/role_assignments?user.id=20bf88e1a4594d4f973c5dd049aeadd4&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:38.226032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951af7-5bbb-487e-a64c-5477cd7f237f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.235067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1233-c44c-41b3-9225-7297891ee593 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.235419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1233-c44c-41b3-9225-7297891ee593 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.235643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1233-c44c-41b3-9225-7297891ee593 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.237133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 259/538] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:25:38.254401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:38.255745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.256196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.256616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.280588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.281272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.281555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.281830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.282561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:38.282870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:38.284349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.296679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:38.297894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:38.313089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 280/539] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:38.318886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.389938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 260/540] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 19:25:38.412588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.499553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac9b8-4c8a-4c26-8522-fd3a3202447d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.573718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:38.581119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.583260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7bd67-6d8e-4b3a-b88b-21ebe97a9ea4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.584272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.584754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.585226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.600155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:38.603704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:38.629798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:38.630704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:38.631573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:38.634032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 281/541] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:38.639649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.660985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.661809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.662078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.662355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.684586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:38.686825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:38.692716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:38.693623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:38.694502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 261/542] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:38.699874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.829910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18d297-68eb-4646-8d33-860c53efdf87 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.842951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9636546-8f4d-4284-9f82-cb7e59e14061 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:38.888043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.888725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.889010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.889290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.890057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:38.890366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:38.892925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.906463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:38.907157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:38.907500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:38.907782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:38.908594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:38.908905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:38.910341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:38.915373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 262/543] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:38.927719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.939466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 282/544] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:38.952472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:38.956549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:39.088986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50ca88-681d-4f69-9bac-a766572b574d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.114364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:39.123032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fc1e3-af7b-40b9-9d0d-af22dfbbdc58 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.133885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.134691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.135000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.135277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.148437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:39.149888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:39.166823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.167523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.167807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.168119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b2b50738-6713-4c36-ab9f-4396d75845dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:39.173342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:39.174193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:39.175078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:39.175078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:39.175078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:39.175078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:39.175388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=bf8b8605da7d49cfbdfb0162cbbf2ff2) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:39.175459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 263/545] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:39.175524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:39.175589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff394-e936-49c3-ad0e-04c8a437e84f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.175589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff394-e936-49c3-ad0e-04c8a437e84f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.175589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff394-e936-49c3-ad0e-04c8a437e84f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.175766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:39.182628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 264/546] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:39.182628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.182628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.182628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.183022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 283/547] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:38 2022] GET /identity/v3/role_assignments?user.id=bf8b8605da7d49cfbdfb0162cbbf2ff2&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:39.183022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.202411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:39.203754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:39.295307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:39.303091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 265/548] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:39.363095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.387182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301caac4-8ab7-4297-b5bb-d7a0ca5b238f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.457668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.458707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.459226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.459767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.460726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:39.461232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:39.462792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:39.492603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 284/549] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:39.498952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab613141-432c-4f8c-8a3a-7218fab12c92 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.502080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.535339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.537315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.537315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.537315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.550288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:39.560395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:39.565724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:39.566641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:39.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:39.575238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 266/550] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:39.584054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.661352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:39.717761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb20a8-629b-4927-9af6-1906fac09a11 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.754655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113a22-a3c1-4a24-96ff-f9f347513d62 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.767071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.767745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.768025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.768317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.769049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:39.769354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:39.770739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:39.785830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 267/551] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:39.791472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.819231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:39.820026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:39.820307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:39.820582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:39.835665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:39.837093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:39.859157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:39.860136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:39.862050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 285/552] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:39.868994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:39.969683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b7cd-c506-4b24-ab2e-e5ceddbbada3 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:39.992444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.016993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.021361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.022167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.022452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.022726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.041964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ad8b7-efd8-4dcb-93b9-256ead1726a1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.047645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.049138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:40.078175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:40.080271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:40.082358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:40.082358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:40.082358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:40.082358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:40.090103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 268/553] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:40.091051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.113760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.114438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.114722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.115034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.116077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:40.116402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.118093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.149945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 286/554] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:25:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:40.153881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.272636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9d2a-c6ee-45a4-9526-8d4cf1f7f6f2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.332501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.333179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.333460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.333729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.334458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:40.334764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.336178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.341251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28f4e2-be6d-451c-adf7-f96de9430c6c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.382103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 269/555] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:40.389323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.414069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.415246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.415763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.416377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.433369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.457149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.543788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.545969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409fe2c-d85c-45c0-bc40-4b18623f9a4e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.571235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e11af-26fe-4fa9-9215-6b9c401c808f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.622861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.624318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.625276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.625995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.636559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.638335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-28c3d216-35e4-4af2-b39a-a140807db255 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:40.659293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:40.660910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:40.662521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:40.669389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 270/556] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:40.669389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c92b-2e87-47d2-8d59-fe0caaca09b4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.669389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c92b-2e87-47d2-8d59-fe0caaca09b4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.669389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c92b-2e87-47d2-8d59-fe0caaca09b4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.670030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 271/557] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:40.674472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae4fa-4676-4575-8f84-0008ed1bf40a None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.674761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae4fa-4676-4575-8f84-0008ed1bf40a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.675164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae4fa-4676-4575-8f84-0008ed1bf40a None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.681074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 272/558] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:40.681290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 287/559] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:40.681464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ae4fa-4676-4575-8f84-0008ed1bf40a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.681631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.681631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.681631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.698758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:40.699999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:40.735681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.767085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 288/560] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:40.767085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.784300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:40.797184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ae4fa-4676-4575-8f84-0008ed1bf40a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.833498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.834328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.834613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.835064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.835912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:40.836227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.837807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400759f-258e-403e-94b5-6ee3fc426ca7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.858841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 273/561] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:40.860581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.861154 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.861624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.878866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:40.880280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:40.881015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a085ff-f15a-4d4e-bfc0-8eb720b469fa None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:40.918405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:40.919231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:40.919512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:40.919788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:40.930224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:40.931693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:40.955653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 274/562] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:40.955653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:40.955884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:40.956715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 289/563] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:40.957595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:40.988648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.040774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.061800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0858735-57cb-4f65-98ec-40ab7529be1c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.074299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdb95c-def2-44af-a5a6-a9ccd655e123 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.099880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.100678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.100962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.101239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.112194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:41.112359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.113297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.113732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.114158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:41.120441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.121629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:41.122647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 275/564] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.123437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.131055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.132781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:41.153031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.154102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.155238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:41.155238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:41.155238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:41.155238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:41.156733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 290/565] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.157710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.235124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.245793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972bfd7-93b1-4f75-bbaa-273e8b23472e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.257527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fb612-1dfa-4efa-b0a4-ad06f291c126 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.281414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.282180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.282467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.282749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.283510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:41.283822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.285424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.293593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.294863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.295178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.295456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.296493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.304089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 276/566] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:41.304089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.310782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:41.318908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.319861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:41.320868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 291/567] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.321557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.425978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f10d-4e29-4526-84b2-430a03618e26 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.437731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd96b6-4014-434e-96cc-8869295947da None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.462734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.463537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.463820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.464099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.465432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:41.465741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.467228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.474988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.475534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 277/568] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:41.475654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.475927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.476205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.476393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.476993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:41.477308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.478671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.481224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.491058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 292/569] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:41.491853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.538703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.577197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb200cd-350e-4443-8412-2dbd4b5f811c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.588282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dfa07-38af-4fe1-9626-5e154e204240 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.610724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.611557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.611846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.612128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.618681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.620154 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.621984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.622635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.622940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.623223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.623958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:41.624261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.625591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004ab9c-45e3-4801-979c-65879b2432e2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.633225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 293/570] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:41.634400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6640b3-2f7d-4c6e-ae35-ac807151875f None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.634688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6640b3-2f7d-4c6e-ae35-ac807151875f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:41.634849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.635807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:41.637289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:41.637894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6640b3-2f7d-4c6e-ae35-ac807151875f None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.637894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 294/572] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:41.637894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6640b3-2f7d-4c6e-ae35-ac807151875f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.638965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 278/571] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.640192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.702628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.744614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65975acd-43a0-4b31-b11f-23fce9d8244b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.745169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6640b3-2f7d-4c6e-ae35-ac807151875f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.768107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.779289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.779698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.780088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.780373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.780483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.780652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.780753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.781027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.787477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.788893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.791030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.792447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4f696c88-3c59-4e62-88c7-5a72f80e0e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:41.803172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.804030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:41.804938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:41.807071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 279/573] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.808472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.808974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.809108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:41.809863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:41.810766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:41.811692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:41.811692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:41.811692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:41.811692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:41.811692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 295/574] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:41.812239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.829469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:41.830690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:41.895168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 280/575] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:41.896002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:41.912664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff4019-8e4d-47c5-8367-985b32a11d69 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:41.923858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:41.946592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:41.947276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:41.947563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:41.947844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:41.948632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:41.948975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:41.950441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:41.962800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 296/576] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:41.963689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.008031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9190f8-d4af-49dc-b61a-0410c4f8d09f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.020838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.042692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.044033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.044319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.044595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.054842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.056309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.062869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104238-0064-432f-b165-6e9fd2d7865b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:42.073650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:42.075162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:42.076076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:42.076076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:42.076076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:42.076076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:42.076076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 281/577] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:42.076364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.098640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.099250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.099534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.099812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.100594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.100897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.102299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.113366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 297/578] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:42.113982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.143688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.176976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d40c-2c36-4b46-8a8f-d83cebfeaf4e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.209717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.210371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.210652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.210955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.211668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.211978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.213340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.219787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5e610-1179-4606-9fbd-31dc8a5f7a67 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.225217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 282/579] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:42.225989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.249521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.255707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.256498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.256780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.257057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.267609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:42.275210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:42.276333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 298/580] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:42.278088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.325910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d4264-16a3-48b2-99a4-8c618fcb89f1 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.359478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.360125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.360414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.360691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.361522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=e8e2bf373b15496597a6f18d14244495) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.361835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.363281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.363725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.369377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 283/581] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/role_assignments?user.id=e8e2bf373b15496597a6f18d14244495&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:42.370130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.373287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8eb96f-8bc4-42a9-844e-316bddb2462e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.415890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.416803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.417195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.417486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.418281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.418610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.420051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.436116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 299/582] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:42.437287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.484444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.498191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a783efb-1d77-4ed0-9c1c-e593cbfc3554 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.538357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733e38f-a755-4596-8d86-743287136813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.539310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e38f-a755-4596-8d86-743287136813 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.539596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e38f-a755-4596-8d86-743287136813 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.539878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733e38f-a755-4596-8d86-743287136813 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.548229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c3a59-8161-43ee-bc7d-01e4abec69a0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.554789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733e38f-a755-4596-8d86-743287136813 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=a82e215bee54480b8e4f69ca6352a865, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.556284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733e38f-a755-4596-8d86-743287136813 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.584207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423da73-9352-46be-b417-478e06cee29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.585000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423da73-9352-46be-b417-478e06cee29d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.585279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423da73-9352-46be-b417-478e06cee29d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.585551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423da73-9352-46be-b417-478e06cee29d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.592925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423da73-9352-46be-b417-478e06cee29d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.594341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423da73-9352-46be-b417-478e06cee29d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2423da73-9352-46be-b417-478e06cee29d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:42.608689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:42.609876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 300/583] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:42.611100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423da73-9352-46be-b417-478e06cee29d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.626278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.633866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 284/584] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:42 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:42.634559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733e38f-a755-4596-8d86-743287136813 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.735142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423da73-9352-46be-b417-478e06cee29d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.776719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733e38f-a755-4596-8d86-743287136813 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.786533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.787361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.787740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.788089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.789445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.789824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.791932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.804176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 301/585] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:42.805194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.823532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.824443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.824728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.825003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.833584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.835141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:42.837580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:42.858676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:42.859924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:42.861083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:42.862951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 285/586] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:42.866945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:42.899484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:42.940527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc54dd-b020-4cde-bf0f-b735a0096eb4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.970753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcabf-7571-4226-ae68-d5ebeb3373f2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:42.981916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:42.982960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:42.983248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:42.983526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:42.984689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:42.985147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:42.986852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.007320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 302/587] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.008168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.014601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.016402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.016688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.016969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.029652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.031867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:43.055338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:43.056640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:43.059058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:43.059058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:43.059058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.059058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:43.066974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 286/588] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:43.066974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.127226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.144681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3d5e-1926-411a-85f4-eb7363379cbf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.189503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.191729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65e91-b074-416e-a224-cf92958666d8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.197231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.198024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.198411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.198789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.200294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:43.200711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.202651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.230996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 303/589] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.232168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.232943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.233809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.234193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.234536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.249318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.251486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:43.274287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:43.275610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:43.276714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:43.276714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:43.276714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.276714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:43.279049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 287/590] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:43.279991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.343193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e64-92c6-4fc8-ab41-08e24134f502 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.379559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.380446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.380893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.381316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.382175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:43.382626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.384163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.395960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 304/591] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.396958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.413019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc98953-de3e-4e9f-90bb-dcd6843ac95f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.414763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.448893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.451893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.452830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.453220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.453600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.468668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.470264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:43.489284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:43.490243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:43.491200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:43.491200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:43.491200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:43.491200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:43.491200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 288/592] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:43.492033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.510510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20e802-c70d-455a-869f-a12dc6e89325 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.547249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.548218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.548830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.549285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.550279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:43.550776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.552398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.567524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 305/593] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.568769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.610770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9c7e2-1d1b-4bba-a0dd-ae91bbeff9e5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.661466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.662312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.662668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.663042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.673283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.675257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:43.691752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:43.692711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:43.693635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:43.695174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7a85-90d2-4de5-a467-ffe071d11d3f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.696517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 289/594] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:43.697481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.709696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.719734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:43.738650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.739668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.740194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.740726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.741681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:43.742272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.744030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.757892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 306/595] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.759304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.837677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742e5f-8c8d-45b2-aceb-8d9cbedee9cd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.870846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159e528-8217-435d-a82a-51754eecda12 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:43.883249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.884047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.884588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.884869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.894876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.897043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.907476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:43.908275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:43.908626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:43.908970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:43.909934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:43.910309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:43.911925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:43.914118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:43.915029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:43.916678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 290/596] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:43.922954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.934510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 307/597] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:43.935675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:43.972532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.016733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.043949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc355b2-a533-44ee-b2e4-892e7b7a986a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.044857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572c49d-306b-424b-9ea3-f26884c94a1f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.081192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.081847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.082136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.082638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.083421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:44.083772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.085136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.088047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.089000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.089434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.090066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.097333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 308/598] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:44.097839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.098244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.099805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:44.115572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:44.116727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:44.117779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:44.119885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 291/599] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:44.121639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.190538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.206850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8707d-0697-40a3-8cc8-5695f3b5b0d2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.231767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f31f-b456-4dc4-932f-c60c7ae27a2b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.242474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.243886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.244341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.244810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.257162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.258699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.269798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.270899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.271358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.271842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/56e334de6eee4d5eb8f2fafdb652e9b5/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.274491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:44.278352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:44.279467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 309/600] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:44.284895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.298456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=56e334de6eee4d5eb8f2fafdb652e9b5, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.300315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.357461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 292/601] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:44 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/56e334de6eee4d5eb8f2fafdb652e9b5/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:44.359066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.426835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41cd26-6f0d-4e3a-8e75-9ae44c80915a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.490621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e798-8c66-472d-865d-f06a354d6240 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.531763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.532446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.532733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.533013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.533420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.533815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:44.534128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.534546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.535020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.535436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.535587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.536433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:44.536884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.539278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.543947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.550258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 310/602] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:44.555744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.562387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 293/603] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:44.563165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.625989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.693459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4298-d321-4473-b5bd-8bc8f170495e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.706649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1e65-0ff3-4304-a0c1-57e503e56e44 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.732647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.733649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.733935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.734210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/20bf88e1a4594d4f973c5dd049aeadd4/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.750949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=20bf88e1a4594d4f973c5dd049aeadd4, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.752422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe46432-4242-45ab-8aad-a208891aed51 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.752604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.753369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.753658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.753943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.762301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.764126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7831f30f-05c4-4032-a598-d7c0e3eb1216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:44.779177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:44.780187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:44.781350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 294/604] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:44.782199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef69d2-6cb9-425b-ae7e-89b672c92f17 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.782199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef69d2-6cb9-425b-ae7e-89b672c92f17 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.782397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef69d2-6cb9-425b-ae7e-89b672c92f17 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.783809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 295/605] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:44.788497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef69d2-6cb9-425b-ae7e-89b672c92f17 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:44.856486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 311/606] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:44 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/20bf88e1a4594d4f973c5dd049aeadd4/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:25:44.858670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9331d9-12b2-43f9-afe6-54e6d45dbece None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.859081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9331d9-12b2-43f9-afe6-54e6d45dbece None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.859435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9331d9-12b2-43f9-afe6-54e6d45dbece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.899153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6f9331d9-12b2-43f9-afe6-54e6d45dbece None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:44.900511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6f9331d9-12b2-43f9-afe6-54e6d45dbece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:44.931783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef69d2-6cb9-425b-ae7e-89b672c92f17 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:44.935250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.962113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:44.981753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:44.983003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.983827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.987668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.987668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:44.987668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:44.987668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:44.995250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 312/607] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:44.996905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd426-4446-4140-8433-5e5f46983f47 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:44.997272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd426-4446-4140-8433-5e5f46983f47 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:44.997559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd426-4446-4140-8433-5e5f46983f47 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:44.999301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 313/608] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:45.004855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.005189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.005430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.012819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 296/609] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:45.018965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.035314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:45.036671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:45.117433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 314/610] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:45.118292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.145055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7ddf-2407-413c-b634-daaaba515717 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.187715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.188615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.188996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.189292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/bf8b8605da7d49cfbdfb0162cbbf2ff2/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.206318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=bf8b8605da7d49cfbdfb0162cbbf2ff2, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:45.207928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:45.220736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f309-03a9-49e8-a60b-8504dc900497 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.230256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:45.255969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.256751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.257037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.257325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.259492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 297/611] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:45 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/bf8b8605da7d49cfbdfb0162cbbf2ff2/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:45.265682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.269609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:45.281236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:45.282266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:45.283717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 315/612] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:45.291536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.300828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:45.402855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f7552-3ebd-4365-bdfe-f1f7e400d359 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.442623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.443465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.443749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.444030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.457096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:45.465369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:45.468348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:45.469197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 298/613] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:45.472642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.498890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa443-6bd0-45c4-a35f-c3a05066ab1d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.547418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.548468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.548962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.549462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.550428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:45.550964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:45.552567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:45.564506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 316/614] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:45.569872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.584781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:45.602096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:45.617612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032c31-cd9d-4d6b-80b4-232a0a229432 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.658821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.659847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.660351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.660805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.661778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:45.662816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:45.664571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:45.678341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 299/615] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:45.680006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.765416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f369e5b-6841-4ecf-9b60-90c1defd61b1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.829699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.830500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.830784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.831085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.835362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0b2-ac0f-4fe9-8f00-55cd5d31d06b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:45.853230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:45.855009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:45.877498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:45.878415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:45.879743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:45.879743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:45.879743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:45.879743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:45.879743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 317/616] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:45.884493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:45.897858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:45.898917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:45.899539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:45.900013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:45.907228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:45.913296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:45.914734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:45.941656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:45.943065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:45.943948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:45.943948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:45.943948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:45.943948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:45.943948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 300/617] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:45.953686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.039221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:46.069247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afc8b-ba17-4603-bed6-5c53fe4f04c0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.135447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6d176-5e4c-4041-837f-16c4d0727cf5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.152205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.152879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.153164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.153436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.154165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:46.154505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.155851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.174977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 318/618] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:46.178676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.182041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.182176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.182176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.182176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.182176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:46.185533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.188123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.217340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 301/619] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:46.222494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.333783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:46.341569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095caae0-69cc-43b5-aeef-a0b0f6955baa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.408692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.409492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.409773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.410048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.422208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.423693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.428742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5026-b48a-4be9-af44-23343ace370a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:46.444916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:46.445938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:46.447266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 319/620] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:46.451697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.463298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:46.514689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.515541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.515828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.516398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.533927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.535434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:46.559781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:46.560784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:46.561737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:46.562521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2328844-7718-40a8-93a2-865e0130abb3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.567369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 302/621] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:46.575897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.597999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.598610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.598901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.599206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.600054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:46.600368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.601800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.622161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 320/622] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:46.633876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.719704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e30ea-2875-4cf6-86c2-e13615a6af48 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.729009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:46.763129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.764247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.765993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.766487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.767617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:46.768388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.770886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.781741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:46.812691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 303/623] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:46.821669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.842819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177db2b-3560-4c01-83f1-4c3ea084b9d4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:46.890640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:46.891386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.891673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.891954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.907951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:46.909709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-96f36b39-ecf6-47dd-971f-cc00884efc55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:46.928736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:46.929690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:46.930582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:46.930582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:46.930582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:46.930582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:46.932976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 321/624] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:46.935075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5f89-d21f-4b07-b1ce-1aa8827ea43d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:46.935587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5f89-d21f-4b07-b1ce-1aa8827ea43d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:46.936059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5f89-d21f-4b07-b1ce-1aa8827ea43d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:46.941756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 322/625] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:46.942760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd5f89-d21f-4b07-b1ce-1aa8827ea43d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:46.971320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f25c-e3f5-40f1-9e94-1cfed8fb27cb None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.015341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.015998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.016286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.016570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.053026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.082623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd5f89-d21f-4b07-b1ce-1aa8827ea43d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.090773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.092513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.099200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.127460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.128288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.128584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.128895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.130727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:47.131277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.132983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f048dd-dc05-4c84-ab76-5b1d2d8be4c7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.151888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 323/626] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:47.153434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f183c6-5877-4f1d-8c06-151578be23f3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.153684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f183c6-5877-4f1d-8c06-151578be23f3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.154041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f183c6-5877-4f1d-8c06-151578be23f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.175061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-36f183c6-5877-4f1d-8c06-151578be23f3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:47.176474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-36f183c6-5877-4f1d-8c06-151578be23f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:47.222313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 304/627] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:47.223081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.253510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 324/628] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:47.255561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c695f-3def-402d-9e60-b05b8b1260f2 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.256300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c695f-3def-402d-9e60-b05b8b1260f2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.256733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c695f-3def-402d-9e60-b05b8b1260f2 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.263637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 325/629] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:47.263637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c695f-3def-402d-9e60-b05b8b1260f2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.293707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.333367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b7c9-1f75-49f7-9bb1-9a13278a1152 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.369858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.370697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.371533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.371813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.378145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c695f-3def-402d-9e60-b05b8b1260f2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.378938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.380218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.380424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-e4a29968-befe-418e-808d-7645467fd28a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:47.394571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:47.398712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 305/630] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.399776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.400550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.416512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:47.418185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:47.420388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.421196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.421472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.421747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.432663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.434995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:47.453777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:47.454689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:47.455728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:47.455728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:47.455728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:47.455728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:47.459298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 326/631] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:47.460139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.491001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 306/632] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:47.492160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.555694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.571653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13aa38-a9cf-4cec-baae-790fedcc90be None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.603899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfc6f-fbda-46fc-b03f-db0e4db9adbd None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.608175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.608975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.609262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.609541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.620240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:47.628755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:47.629670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:47.630516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 327/633] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:47.631261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.645127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.645895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.646079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.646360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.647197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:47.647512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.647663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.648965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.666502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 307/634] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:47.666502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.747603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba459d-401f-4390-8394-f557156a8398 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.776218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e291a5-3d30-4d6c-ad7f-53a9f085fa0a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.785922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432245e7-8033-409f-8007-450fee18758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.787022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432245e7-8033-409f-8007-450fee18758a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.787460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432245e7-8033-409f-8007-450fee18758a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.787897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432245e7-8033-409f-8007-450fee18758a None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.788812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432245e7-8033-409f-8007-450fee18758a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:47.789126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432245e7-8033-409f-8007-450fee18758a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.790596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432245e7-8033-409f-8007-450fee18758a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.796081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.805829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 328/635] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:47.806549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432245e7-8033-409f-8007-450fee18758a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.823416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.824355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.824645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.824929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.835234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:47.843602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:47.844750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:47.845619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:47.850251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 308/636] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:47.851133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.889933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:47.913821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432245e7-8033-409f-8007-450fee18758a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.948893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.949562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.949850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.950129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:47.950223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4c999-e979-40ba-a20e-ac3f9fb09cf9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:47.950880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:47.951212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:47.952576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:47.960709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 329/637] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:47.961527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:47.989203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:47.990099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:47.990379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:47.990653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/e8e2bf373b15496597a6f18d14244495/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.005777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=e8e2bf373b15496597a6f18d14244495, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.007200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.057952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41404d6-8b15-4e95-ace5-aecb24ae56a3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.078301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 309/638] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:25:47 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/e8e2bf373b15496597a6f18d14244495/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:25:48.079174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.095262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.097075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.097366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.097668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.100989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.107343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.108879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.122952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-d1d4842e-100d-4b66-a3f8-f5a7ca7d35bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:48.124937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:48.125999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:48.127015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 330/639] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:48.128008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734751a-76cf-44f3-999d-e65812a9575d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.128302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734751a-76cf-44f3-999d-e65812a9575d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.128584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734751a-76cf-44f3-999d-e65812a9575d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.130147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 331/640] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:48.135248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734751a-76cf-44f3-999d-e65812a9575d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.222417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b2c6-8ee2-4e6e-b6f2-2a326221caa2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.253935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734751a-76cf-44f3-999d-e65812a9575d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.262311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.262940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.263230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.263504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.264630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:48.264943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.266631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.279161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 310/641] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:48.279161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.292386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.293529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.293914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.294258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.304422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.305930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-ff78a1b8-2526-46fe-8000-9f783c4cb81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:48.325378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:48.326367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 332/642] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:48.327979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.328340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.328340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.346690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:48.348135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:48.373411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.385016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194e0d9-d01b-4e65-9a66-d3ac63321d6d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.394243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.418016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 333/643] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:48.419147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.431379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.432305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.432773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.433773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.434763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:48.435273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.437165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.450789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 311/644] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:48.451937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.540481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20313f-9b62-423a-8643-1036d9dde919 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.553198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b573e4-aef1-44d5-8697-8b3198360c85 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.579958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.580906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.581209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.581784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.588896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.589596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.589882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.590165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.590895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:48.591242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.592627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.594143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.596179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.604347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 312/645] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:48.605196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.616627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:48.619846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:48.621281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:48.622327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:48.622327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:48.622327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:48.622327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:48.622327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 334/646] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:48.622612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.659247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.714593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4f5d-2bcb-481c-be1b-064cbdabfc54 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.737567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e5b9-addf-40ce-832e-e197671ac1c2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.749198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.749995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.750280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.750554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.760658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:48.768924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:48.769943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:48.770832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 313/647] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:48.771622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.777148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.777802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.778085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.778360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.779234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:48.779587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.781072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.799282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 335/648] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:48.800128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.860460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:48.906642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b47a-263b-475b-b332-aad424d5bed2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.915428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475dc84-fddf-475e-9f40-2ef6bc58c94b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:48.954037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.955026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.956250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.956250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.959031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:48.959903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:48.960400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:48.960866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:48.961799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:48.962287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.964743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.966408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:48.968909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:48.978479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 336/649] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:48.981602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:48.985399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:48.986362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:25:48.987450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:48.991072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 314/650] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:48.992119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.005832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.113685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d66-0c09-4ce0-8219-e05464719478 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.117883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f632e7-dd8d-4b66-a35b-ff1f630d79a9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.155590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.160889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.164762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.165128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.165471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.166260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:49.172142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.172142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.172369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.172369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.172369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.172369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.182705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.185332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.185687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 337/651] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:49.186639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:49.205916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:49.206948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 19:25:49.207912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:49.210600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.211271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 315/652] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:49.212980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.314019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95853a-06ba-4454-b980-19446d0aafdb None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.328989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d60c-190c-4883-b89c-ea4aa11ef52f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.361346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.362287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.362734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.363191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.364238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:49.364704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.365116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.366000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.366383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.366755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.367052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.378211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.380136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.393268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 338/653] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:49.393268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:49.406971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:49.408081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:49.409156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:49.409156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:49.409156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:49.409156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:49.416380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 316/654] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:49.417060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.471502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.501934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.539110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c6f7-7096-417c-9ee1-a5da465ff41c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.539865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d6a78-33d6-4e69-9caa-1ca330490f10 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.579150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.580229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.580737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.581217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.581786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.582566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.582968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.583257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.584100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:49.584411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.585957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.605208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 339/655] 10.222.0.36 () {56 vars in 1137 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:49.606157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.657492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.659514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.704344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.711640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafe6c4-ae0b-4dd5-b6ce-39c1b3eb1c14 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.743411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.750825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.751774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.753006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.753292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.754033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:49.754342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.755962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.778731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 340/657] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:49.779097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 317/657] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:49.780309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.780860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.888316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc2c6-3f1c-404e-84be-c98fd28635ce None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.894111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a243b-9eef-4481-872a-dcdb2d6f5cdb None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:49.924185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.924845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.925194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.925636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.931526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:49.932564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:49.933572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:49.934048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:49.942464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:49.944321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:49.947637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:49.961708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:49.962696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 19:25:49.963690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:49.966499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 318/658] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:49.967557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:49.996688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.008638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.010202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.081575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5212-0d5a-4c66-9f79-c61b1f9ae602 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.118546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.119734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.120254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.121289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.124202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 341/659] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:50.125274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.133675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.135507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:50.156126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:50.157107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:50.158030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:50.158030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:50.158030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:50.158030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:50.165408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 319/660] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:50.166806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.228911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.239265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.248608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2c600-a26e-4e35-8ffa-472c82185831 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.277493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645d2b8-906b-47e8-8367-303fdc8bc7ec None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.284546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.285219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.285505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.285783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.286749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:50.287091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.288886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.313308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 342/661] 10.222.0.36 () {56 vars in 1139 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:50.313308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.315584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.317304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.317618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.317897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.329658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.331232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3f268eea-f53c-4257-a962-749adec60d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:50.350046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:50.351226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:50.352582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:50.352582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:50.352582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:50.352582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:50.356377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 320/662] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:50.357644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335711-ca40-41f4-9037-0a63dd505ade None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.358045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335711-ca40-41f4-9037-0a63dd505ade None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.358293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335711-ca40-41f4-9037-0a63dd505ade None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.362978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 321/663] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:50.362978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af335711-ca40-41f4-9037-0a63dd505ade None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.419427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e920c-8070-4837-b252-8642f22b337d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.457141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.457815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.458139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.458415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.459132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:50.459500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.460848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.467286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af335711-ca40-41f4-9037-0a63dd505ade None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.471161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.474188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 343/664] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:50.475937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.491339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.506337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.507317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.507817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.508488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.580550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.582275 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.582852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595c78a-3ba9-404c-933e-e2bf0615a1bb None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.620360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.621599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.621890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.622165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.622883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:50.623221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.624579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f07e85-a977-4b4b-80e3-ecf7bbfb1629 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.636766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 344/665] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:50.637813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.638108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.638388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.660150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:50.661435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:50.692885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 322/666] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:50.698266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.702725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.731385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.747030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 345/667] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:50.747030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.810831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd182e07-454b-4d61-8028-e14391ca2268 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.845920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.846706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.847015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.847291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.847401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da66cdd-3640-416c-a675-99436eaeafd5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:50.854489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.856528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:50.871414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:50.872341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:50.873240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 323/668] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:50.874283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.887275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:50.888076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:50.888365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:50.888652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:50.895412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:50.896884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:50.911829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:50.912915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 346/669] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:50.913812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:50.942138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.978404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:50.983194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa675-35ac-4426-a719-9c56d4f51a4e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.020612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.021902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.022025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c976cc-0ed9-4b73-af6f-06b897a55794 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.022506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.023065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.035273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.037181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.060079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:51.060293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:51.061150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:51.063087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:51.063087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:51.063087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:51.063087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:51.063331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.063331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.063331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.063523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 324/670] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:51.063523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.078000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:51.086302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:51.087487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:51.088409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 347/671] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:51.089186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.171193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b36d3e-cbf3-4d6a-80b2-48d1174948d8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.189304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.191254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af8a0d-7032-4a14-8449-25ebc7390499 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.206439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.217978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.218463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.226568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 325/672] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.226568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.226830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.226830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.226830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.226830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.227168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.227168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.227437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe296f-13a7-48bf-a4ea-5add79b4a3da None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.239740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 348/673] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.240983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72809b9-edbf-4e5e-bc85-d6289144efe9 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.241271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72809b9-edbf-4e5e-bc85-d6289144efe9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.241549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72809b9-edbf-4e5e-bc85-d6289144efe9 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.242940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 349/674] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:51.247506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72809b9-edbf-4e5e-bc85-d6289144efe9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.320824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70740d-907e-45ce-abcf-060b30466c62 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.346100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72809b9-edbf-4e5e-bc85-d6289144efe9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.353144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.353802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.354087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.354366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.355656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.355965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.357334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.368704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 326/675] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.369487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.384209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.384876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.385159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.385434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.386211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.386518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.387900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.395712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 350/676] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.396487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.402467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.442241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.467722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7a485-c27d-4b36-a34e-e2d8eae3fd8b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.497959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9add-af8c-4bcc-b48f-41614d248a65 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.501197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.501848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.502139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.502422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.503333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=56e334de6eee4d5eb8f2fafdb652e9b5) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.503655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.505096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d96da-858b-4d7c-82d0-4d47fb3acdd2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.510808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 327/677] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/role_assignments?user.id=56e334de6eee4d5eb8f2fafdb652e9b5&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.512189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.512472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.512684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.532305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.532994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.533281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.533562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.534423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=20bf88e1a4594d4f973c5dd049aeadd4) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:51.534745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.536268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.554969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 351/678] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/role_assignments?user.id=20bf88e1a4594d4f973c5dd049aeadd4&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:51.555197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:51.555318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.555410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:51.643217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 328/679] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:51.644315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.691171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.708036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6dd7-3138-4b3e-bcbc-002e851fad78 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.717099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.751177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.751999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.752339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.752573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.761893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.763675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.779661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fe9f3-15d7-435f-a8a5-aa8eb91a7d89 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:51.780973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:51.781857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:51.782772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:51.785678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 352/680] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:51.787438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.820982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.821676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.821961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.822450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.840673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:51.842126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:51.862488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:51.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:51.864577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:51.864577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:51.864577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:51.864577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:51.864577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 329/681] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:51.865251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:51.918971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59676d8-5fd4-4dea-ba6a-5a27b138c7ca None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.953705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:51.957384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:51.958368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:51.958829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:51.959343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:51.971356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:51.974578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa06c7-8c46-4196-ad64-df8ec40aae0b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:51.979443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:51.981188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:51.982324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:51.984944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 353/682] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:51.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.016192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.032281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.033311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.033600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.033877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.035018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.035347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.037126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.060286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 330/683] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.060286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.108416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c601f-66cd-4d33-ad1f-eddbcfba12c5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.149510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.150187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.150472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.150853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.151787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.152106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.153494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.168884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 354/684] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.169939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.173874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70180474-70e2-46d0-bc9d-f7bd415e3c67 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.209157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.209831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.210121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.210404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.211209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.211528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.212904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.222281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 331/685] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.223147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.226538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.266634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.282853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d63f7c-d285-464b-9125-35a4b17cd27e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.318811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.319525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.319817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.320096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.320933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=bf8b8605da7d49cfbdfb0162cbbf2ff2) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.321248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.322676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.328836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d5381-3bdc-46c7-bee9-2aba6fc0772b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.329458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 355/686] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/role_assignments?user.id=bf8b8605da7d49cfbdfb0162cbbf2ff2&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.330358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.366957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.367795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.368169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.368348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.377879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.379381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:52.394520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:52.395566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:52.396527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 332/687] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:52.408043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.449680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a1f9-9629-4bd8-a8e3-eb4f6b0b3273 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.485586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.488464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.488464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.488464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.503046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.504533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:52.525780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:52.527350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:52.528230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:52.528230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:52.528230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:52.528230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:52.532798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 356/688] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:52.534440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.538255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.541593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.599967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f4e9-1098-47fb-9f07-860b9d6b86bd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.647526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.648202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.648487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.648768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.649890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.650203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.652999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.672342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678576-7562-487d-8181-5e1027d11512 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.676671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 333/689] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.682740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.713673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.715247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.715734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.716307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.717234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:52.717885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.719577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.734680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 357/690] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:52.739823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.809815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.815467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd8bde-7ff1-4383-b5e7-15ab37bc2c08 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.843338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:52.852920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621d804-35f3-4cf5-9a2d-dac0751f31aa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:52.856115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.857281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.857281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.857484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.891207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:52.892457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:52.892950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:52.893443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:52.902382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.904173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:52.920110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:52.921112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:52.922115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:52.924881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 358/691] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:52.930388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:52.934769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:52.936344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.043784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 334/692] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:53.049174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.074636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.141717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073f571-f319-4303-9b0d-5cfc6f328fc1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.151643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1f9f8-e18c-4b4e-8f8a-397dfd7b780c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.201009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.201883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.202167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.202442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.208398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.216311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.216311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:53.232490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:53.233645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:53.234542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:53.234542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:53.234542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:53.234542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:53.234542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 335/693] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:53.239338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.243585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.244569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.245051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.246050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.247116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:53.247648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.255211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.275764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 359/694] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:25:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:53.281950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.351977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.377372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08b210-135f-485d-a3de-0350b6510715 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.416123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d98d2e-5cf0-4663-9a5a-22628f5d63e8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.443679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.444363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.444645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.444914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.445657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:53.445966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.447376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.455905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.456829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.457438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.457911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.473584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 336/695] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:53.478723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.512060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.532998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.534894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.638487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 360/696] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 19:25:53.644462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.673428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaea5b-bccc-45dc-80c7-696b6f942946 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.751842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.755464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbd8a-4f5d-485c-8f49-8689d28ea74d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:53.759679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.760498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.760786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.761098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.770853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:53.788116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.789580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.804312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:53.805571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:53.806084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:53.806525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:53.827941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:53.828904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:53.829785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:53.832995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 337/697] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:53.834449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:53.838967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:53.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:53.867870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:53.869042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:53.870165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:53.870165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:53.870165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:53.870165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:53.875328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 361/698] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:53.882389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.014616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871561a3-0085-42b7-970e-fa674c4f8dd9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.039281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb35423-c049-4262-b686-60be98fea083 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.052767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.053535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.053885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.054233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.055091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.055467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.057442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.059303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.072283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 362/699] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.079116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.124852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.125517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.125793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.126065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.126858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.127192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.128636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b7b84-47a9-4f49-88fc-db3bf99f062a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.153193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 338/700] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.154684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa26a-304c-4d39-b7fd-e189fe258145 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.155007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa26a-304c-4d39-b7fd-e189fe258145 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.155296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa26a-304c-4d39-b7fd-e189fe258145 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.160502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 339/701] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:54.164010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa26a-304c-4d39-b7fd-e189fe258145 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.225991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.230476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640d7d4-74a5-48b3-a621-526624da13a8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.281306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.283556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.284149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.284644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.285885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa26a-304c-4d39-b7fd-e189fe258145 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.294185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.296423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-100af6cb-db2f-4c0a-990a-556857c6acde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:54.315296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:54.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:54.317840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 363/702] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:54.318349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb5bf-3e44-49b0-9caf-632216a54bd6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.318349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb5bf-3e44-49b0-9caf-632216a54bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.318471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb5bf-3e44-49b0-9caf-632216a54bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.331529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.332189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.332472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.332753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.333597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=e8e2bf373b15496597a6f18d14244495) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.333916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.335382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5465f0e-08ec-4360-b9ba-9ecc3eafb1fc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.336173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c3adb5bf-3e44-49b0-9caf-632216a54bd6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:54.337491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c3adb5bf-3e44-49b0-9caf-632216a54bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:54.342826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 340/703] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/role_assignments?user.id=e8e2bf373b15496597a6f18d14244495&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.344094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c8b0-a37c-4465-a7ec-2f1cf65feaf6 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.344381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c8b0-a37c-4465-a7ec-2f1cf65feaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.344655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c8b0-a37c-4465-a7ec-2f1cf65feaf6 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.346581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 341/704] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:54.347375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c8b0-a37c-4465-a7ec-2f1cf65feaf6 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.397360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.420294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 364/705] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:54.422878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.422878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.422878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.442744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:54.444346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:54.499668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c8b0-a37c-4465-a7ec-2f1cf65feaf6 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.522248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 365/706] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:54.523447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.542358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.561980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.563001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.563558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.563847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.564643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.564958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.567518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.583805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 342/707] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.584908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.651071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be1599-c523-4840-8130-7c24de069fd4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.667000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.700988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.701893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.702192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.702473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.703294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.704773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.707502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a528-f7c3-4a42-b85c-f09b623eb412 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:54.720085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:54.721143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:54.722029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 366/708] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:54.722962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.747284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.748718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.749012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.749292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.764486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.767129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:54.786284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:54.788406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 343/709] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:54.790598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.823734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.838717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c158-d049-489a-9c52-edfd9dbbd142 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.880335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.881007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.881296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.881579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.883312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.883640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.885415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.899141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 367/710] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.900326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:54.904343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac405-2dae-4cab-9479-c7ce14815c93 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:54.943886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:54.944560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.944847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.945123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.945848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:54.946158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:54.947591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e8130-8ded-4c27-8e98-a1190852fb61 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:54.956414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:54.957795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 344/711] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:54.959405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb765-7522-4c82-a6d2-b35dbfcd49c2 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:54.959705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb765-7522-4c82-a6d2-b35dbfcd49c2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:54.959988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb765-7522-4c82-a6d2-b35dbfcd49c2 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:54.961987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 345/712] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:54.963095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb765-7522-4c82-a6d2-b35dbfcd49c2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:55.027342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0bd98-bdf4-42da-96da-3846559b1d7a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.067790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.067790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.067790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.068364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.068744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:55.069054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.070438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642016f-08be-4f84-a282-bc8691f404dc None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.086501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 368/713] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:55.087117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:55.088917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.089777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.090058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.097356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb765-7522-4c82-a6d2-b35dbfcd49c2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.107101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:55.108562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:55.133556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.134414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.134803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.135230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.136537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:55.136919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.138417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.144499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 346/714] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:25:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:55.147724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:55.202318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 369/715] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:55.224985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:55.269993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:55.336101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c47842-161b-475d-848a-d7016ecc3f86 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.373698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.374440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.374723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.375043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] PATH_INFO: `/v3/services/network` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.375701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.377081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:55.389768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:55.390773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:55.392083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 370/716] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:55 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:25:55.396852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:55.399363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211fa9f-b89f-4bfd-a7f1-433e8621598f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.452084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.453615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.454110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.454639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.455621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.457365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f5fa5-862f-4a6b-b1ab-bb7d662dffc2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.515121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:55.521921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9022f0-78d9-4171-8b7a-bc5e18151078 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.557182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.558426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.558714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.559047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.559762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:55.560073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.562468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.566599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 371/717] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:25:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:55.571328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 347/718] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:25:55 2022] POST /identity/v3/users => generated 254 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:55.572567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:55.682515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bd3e9-5d6e-433d-8319-6c89af27a5a6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:55.720098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:55.720788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:55.721074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:55.721353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:55.722073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:55.722454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:55.723931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfaeaf-9018-4482-a75b-590b1365faa0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:55.731371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 372/719] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:25:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:56.771465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738210c2-d0ca-4bca-bcb9-20c5b4f0a193 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:56.772506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738210c2-d0ca-4bca-bcb9-20c5b4f0a193 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:56.773002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738210c2-d0ca-4bca-bcb9-20c5b4f0a193 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:56.774822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 348/720] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:56.780314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:56.781105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:56.781607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:56.798470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:56.799996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:56.817842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:56.900516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 373/721] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:56.944434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738210c2-d0ca-4bca-bcb9-20c5b4f0a193 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:56.964076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:57.076627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738210c2-d0ca-4bca-bcb9-20c5b4f0a193 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:57.119941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:57.121378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.121658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.121933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.135929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:57.143590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:57.144534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:57.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:57.149538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 349/722] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:57.150957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:57.306743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46cf65-924b-47a6-8a59-f1e5a281b1fb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:57.356893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:57.357516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.357802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.358079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.358843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:57.359176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:57.360547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4901d-e259-4d95-a04b-ddc4d2d10667 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:57.375864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 374/723] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:57.381307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:57.480421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:57.508490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d0a8e-b85f-4df5-b36d-3701b2a9288f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:57.553384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:57.554553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.555063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.555534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.563107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d431b8-0b5f-44c1-810d-0f45abf08d74 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.563258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d431b8-0b5f-44c1-810d-0f45abf08d74 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.563485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d431b8-0b5f-44c1-810d-0f45abf08d74 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.565112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 375/724] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:57.567920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:57.569615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:57.569901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.570194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.570473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:57.588244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:57.589840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:57.590735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:57.590735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:57.590735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:57.590735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:57.592966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 350/725] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:57.598061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:57.598504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:57.599422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:57.616316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:57.725315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8a367-39cc-4c80-81c2-af133d16292f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:57.762300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 376/726] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:57.772416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:57.786474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:57.789229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:57.789680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.789680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.789680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:57.789680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:57.789680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:57.791496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:57.819480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 351/727] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:25:57.831989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:57.947377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acd306-c149-4f19-9486-625f1cf080de None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:57.988857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:57.989668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:57.989954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:57.990232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.010097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:58.018326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:58.019289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:25:58.020168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:25:58.025367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 377/728] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:58.030750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1bef-4c73-45e7-8fd8-122b38107149 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.032920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.037496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.108952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.109756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.110043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.110321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.121823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.123304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-dfe4d751-bb3c-4d34-862c-a418086fc04d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:58.145620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:58.148003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 352/729] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:25:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b74e-f7b2-4450-9bc0-0e3dc3b156e6 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.150242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b74e-f7b2-4450-9bc0-0e3dc3b156e6 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.155058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b74e-f7b2-4450-9bc0-0e3dc3b156e6 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.155058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 353/730] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:58.155058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8b74e-f7b2-4450-9bc0-0e3dc3b156e6 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.189587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd74b3f-2188-4e8b-902a-aa066368ba3c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.207739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.228941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.229819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.230106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.230386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.231412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:58.231731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.233478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c5e3d-51ca-4aee-b219-54685051f5dd None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.242160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 378/731] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:58.243467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c8a4-2a93-4f04-85f8-021e6265f42d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.243754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c8a4-2a93-4f04-85f8-021e6265f42d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.244033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c8a4-2a93-4f04-85f8-021e6265f42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.261418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d422c8a4-2a93-4f04-85f8-021e6265f42d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:58.262605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d422c8a4-2a93-4f04-85f8-021e6265f42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:58.270891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8b74e-f7b2-4450-9bc0-0e3dc3b156e6 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.307366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.308566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.308850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.309119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.309951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:58.310254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.312213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.325702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.337659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 354/732] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 379/733] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcf85-0e73-477c-9cef-ffd55918feb7 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcf85-0e73-477c-9cef-ffd55918feb7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcf85-0e73-477c-9cef-ffd55918feb7 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 380/734] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:58.337824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dcf85-0e73-477c-9cef-ffd55918feb7 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.341207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.448913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfda1c-2336-466c-8809-291505775f99 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.451731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.456124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dcf85-0e73-477c-9cef-ffd55918feb7 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.486494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.487566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.487859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.488135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.500749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.501255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.501934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.502216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.502326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.502495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-90c80742-bcb3-4485-a79a-3385d0bf41df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:58.521295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:58.522355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:58.523291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:58.523291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:58.523291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:58.523291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:58.523291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 355/735] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:58.524038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.524136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.524644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.542913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:58.544116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:58.575899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.582269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.583736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.608756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 356/736] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 19:25:58.609556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.683387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 381/737] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 19:25:58.684273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.693909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.708413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37edd9-de2f-4e27-b8c1-ea58e1d1dac8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.742462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.743078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.743359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.743635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.744351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.746271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f29eb4-14cd-45fb-b157-1d9a52d8730b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.773593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:58.779187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7d084-42fc-44e8-881f-79e69c9428f2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.812243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.812941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.813236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.813515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.814233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:58.814542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.815919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:58.826787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 357/738] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:25:58 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:58.827246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 382/739] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:58.827439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.828170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389b51-37d5-4e10-a222-c754fc56c17f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.828445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389b51-37d5-4e10-a222-c754fc56c17f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.828731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389b51-37d5-4e10-a222-c754fc56c17f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.830981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 358/740] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:25:58.831110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389b51-37d5-4e10-a222-c754fc56c17f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:58.949726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcdb27-83ba-44c5-9a1f-c9f27ce37881 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.967520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389b51-37d5-4e10-a222-c754fc56c17f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:58.987795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:58.988881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:58.989232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:58.989583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:58.990326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:58.991780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44090164-3eff-4985-9020-8e7ddc52f697 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.018295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.018295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.018295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.018295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.018730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.018730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.019236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:59.036674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:59.037643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:59.038530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:59.038530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:59.038530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:59.038530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:59.038530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 359/741] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:59.038823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.079087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 383/742] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:25:58 2022] POST /identity/v3/services => generated 240 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:25:59.080885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.081390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.081896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.103479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:25:59.105826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:25:59.121721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.211699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 384/743] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:25:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:25:59.213155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.303197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda058c-edc8-4c27-a99f-3cefb19e1a91 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.334457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063174d-091a-4f0c-8ab8-1079214756a1 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.344768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.345802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.345947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.346104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.356272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.357785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:25:59.373694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:59.374691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:59.375719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 360/744] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:59.376312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.378509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.379298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.379662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.380010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.380833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:59.381215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.382658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.396397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 385/745] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:59.397324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.405311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.439290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.488585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eed6-c679-4072-9c17-ce08366b1b3e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.540004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465771-5917-4af0-bd6b-5dc5ee7d7fd4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.540697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.541491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.541891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.542173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.553334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:25:59.561394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:59.562615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:59.563518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 361/746] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:59.564230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.578442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.579269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.579669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.580008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.581066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:59.581438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.583024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.607223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 386/747] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:59.608247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.669748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.670756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322908d-8e11-4594-8075-9e17043f7d75 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.699358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.714459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c939a-bc3f-4f39-8c92-9aca122ac7bb None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.716656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.717747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.718030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.718312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.727285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.729489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:25:59.749061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:59.751318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:59.753625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 362/748] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:59.754771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.754771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.754771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.754771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.762966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:59.762966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.763205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.763311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.770581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 387/749] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:59.771748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.871006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93ccdb-09ce-4181-ba67-39dca42e967f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.898555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262b07-efc1-40e6-9a24-c17d1c23ef76 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:25:59.910758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.911725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.912145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.912561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.922752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:25:59.923914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.925508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.935486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:25:59.936163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:25:59.936448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:25:59.951866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:25:59.951866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:25:59.951866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:25:59.951866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:25:59.952501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:25:59.954524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 363/750] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:25:59.956670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.957417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 388/751] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:25:59.957417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:25:59.981991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.061229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33679d40-9616-4112-9e83-1cdc7fbd6f91 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.062773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28d36-dc9b-4827-91cf-9f26a5650368 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.097848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.098637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.098915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.099276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.103013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.103688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.103973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.104251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.105509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:00.105819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.107726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.110354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.112112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.125547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 389/752] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:00.126634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:00.136753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:00.137692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 364/753] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:00.139239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.188936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.222181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.246405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8d05a-129a-4c6b-bdd1-9b4a4b3b9338 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.250018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa705d3-72d4-4ddc-ae04-ceca15697f65 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.283741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.284583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.284859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.285133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.289093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.289980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.290490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.291027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.292097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=e8e2bf373b15496597a6f18d14244495) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:00.292414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.292831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.293832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.294897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.301380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 390/754] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/role_assignments?user.id=e8e2bf373b15496597a6f18d14244495&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:00.302463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:00.308020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:00.309066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:00.310436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:00.314557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 365/755] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:00.315729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.421997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade8d5b-57a4-4a41-bab8-27215ab52d1e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.428572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d524-9f59-4ad2-83df-e957c71a36ce None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.431165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.462121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.463159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.464489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.465049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.466121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:00.466766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.468089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.468704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.468833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.469169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.469588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.471212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:00.471876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.474626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.484828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 391/756] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:00.486116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.493793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 366/757] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:00.494813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.499782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.609998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737edc9f-fe7f-46cb-97b9-eb22722b3468 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.644160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.645215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.645594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.645994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.648103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=a82e215bee54480b8e4f69ca6352a865, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.649635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.680793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64287c-d009-4e94-a685-319e7aaebfc0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.719666 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 367/758] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:00 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:00.727836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.728675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.729015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.729354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.738162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.744560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:00.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:00.771018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:00.771957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 392/759] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:00.777421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:00.865899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:00.925123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33578-2beb-4c9b-98b7-574b715c1423 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:00.961599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:00.962202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:00.962494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:00.962776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:00.963686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:00.964573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:00.966002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:00.984906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 368/760] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:00.990141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.021394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.103016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a8a41-c9fe-498f-9e6f-6302ffc257b5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:01.138155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:01.138824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.139215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.139501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.221254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:01.222719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff171-0d1c-4c68-ab2a-b23e9957f62b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:01.251526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.330407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 393/761] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:01.335788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.397519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3d8e-ee95-4a6e-8b10-6da0397f121f None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.397976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3d8e-ee95-4a6e-8b10-6da0397f121f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.398495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3d8e-ee95-4a6e-8b10-6da0397f121f None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.400921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 394/762] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:01.407630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.408043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.408434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.428779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:01.430326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:01.481136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b741bd-d959-48e0-81fc-8e2b440ea095 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:01.499446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.502311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 395/763] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:01.526173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:01.527459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.528058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.528534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.544647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:01.546378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:01.562352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:01.593741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:01.594693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:01.595622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:01.595622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:01.595622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:01.595622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:01.605217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 369/764] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:01.610630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.638714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.699157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff562a-9a18-468b-ab34-a18644c532d2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:01.743772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:01.744778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.746181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.746473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.747488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:01.748869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:01.753007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62488c-d8db-46a3-aee8-69214ed0854f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-162f6103-6917-4d99-9e22-125e4316cc77 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:01.762005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:01.762973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:01.763831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 396/765] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:01.765197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3327-d444-41dd-a02e-509136559a9e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.765197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3327-d444-41dd-a02e-509136559a9e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.765467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3327-d444-41dd-a02e-509136559a9e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.770677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 397/766] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:01.770677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef3327-d444-41dd-a02e-509136559a9e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.799078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:01.800285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.800783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.801269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.802336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:01.802853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:01.804603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275cc09-5411-4bd8-bfd0-b311b347a30d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:01.823228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 370/767] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:01.824988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.825505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.826093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.847954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:01.849505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:01.875676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.893748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef3327-d444-41dd-a02e-509136559a9e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:01.924806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:01.936904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 371/768] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:01.938516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:01.941294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:01.941977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:01.942955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:01.942955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:01.943330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:01.943584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:01.945028 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:01.950331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 398/769] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:01.955843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.049329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c446d-e51b-4a18-8c96-50c3b2bdebbd None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.071297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521843ee-645f-48c3-b55c-6779f785f306 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.087312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.088516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.089656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.090213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.097915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.099735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.108604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.109276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.109609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.109895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.110609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.110937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.112306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:02.117196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:02.118091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:02.119027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:02.119477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 399/770] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.119477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.125797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 372/771] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:02.126837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.147696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.178320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.238257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7607d-a3bc-4954-a1aa-9fd75170edb5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.248801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43d69-0cee-4b7e-a24d-feae9defe794 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.273022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.273973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.274377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.274769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] PATH_INFO: `/v3/services/network` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.275605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.277628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.286477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.287373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.287811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.288238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.289172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] RBAC: Adding query filter params (interface=public, service_id=21280831cdba4accacc8b410529b2776, region_id=RegionOne) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.289634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:02.290569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:02.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:02.293129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:02.293882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 400/772] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:02.294724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.295365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.303877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 373/773] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/endpoints?service_id=21280831cdba4accacc8b410529b2776&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.304975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.393814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.399239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f15e-52b0-4a21-9646-fc7a5cbaf681 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.413041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.421616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e0e6a-66ce-4abc-8bd6-3df33d5765ba None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.434428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.435150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.435441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.435724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.436849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.437167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.438596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.456887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 401/774] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.456887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.468016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.468016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.468016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.468016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.468332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] RBAC: Adding query filter params (name=network) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.468332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.468332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.468332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 374/775] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.468332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.562647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816caf36-c61c-410f-93d5-aea77139d088 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.578345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0f9b0-3bd2-44f1-ba0b-20fd1dff3fab None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.598942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.599645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.600039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.600328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.601024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.602738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.608248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 402/776] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.609131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.618458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.619569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.620097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.620596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.621762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=1e5c2a70586143ac94e5be00608aa6d8) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.622314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.624042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.639182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 375/777] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/role_assignments?user.id=1e5c2a70586143ac94e5be00608aa6d8&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.649890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.729969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bb1c0-b133-4af1-aacf-5a617c663e51 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.770571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.771379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.771668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.771950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.772702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.773015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.774452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d5526-b7a2-4982-9fd8-a07b0dcaa9a0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.781002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 403/778] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.786443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.814537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:02.879108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ad91-32f4-4f47-9237-9377f8fb0455 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.879733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ad91-32f4-4f47-9237-9377f8fb0455 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.879733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ad91-32f4-4f47-9237-9377f8fb0455 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.882065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 404/779] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:02.886533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.886821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.887128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.907563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:02.908750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:02.924684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abda4e5-f10c-4acf-8bbd-01fadb3eb409 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:02.963427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:02.964951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:02.965453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:02.965827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:02.966732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] RBAC: Adding query filter params (interface=public, service_id=eb3fa7a8d1a84cb8970959846c1dd28c, region_id=RegionOne) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:02.967172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:02.968639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:02.974375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 376/780] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/endpoints?service_id=eb3fa7a8d1a84cb8970959846c1dd28c&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:02.979626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:02.999349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 405/781] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:03.025987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.061721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.136685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.161000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60def3-51c3-410e-baee-7b6956a63158 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.198556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7131b-eeae-4d73-898c-fbc6ae7263ad None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.237284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.239075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.239509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.240190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.249275 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.250320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.250783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.251510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.252370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:03.253920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:03.254360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3304a7-1cb6-4e41-b406-ebfd83f439ba None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:03.262736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:03.263740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:03.264605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:03.267368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 377/782] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:02 2022] GET /identity/v3/services => generated 772 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:03.267570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 406/783] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:03.272503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34637-5f68-498a-a6f6-a77e45ab25f7 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.272503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34637-5f68-498a-a6f6-a77e45ab25f7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.272503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34637-5f68-498a-a6f6-a77e45ab25f7 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.272503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 378/784] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:03.274030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.282582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.282870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.283178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.322716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:03.324475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:03.428914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 379/785] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 16 19:26:03.456080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.465202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.483523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c96-a517-43d3-821e-105ae4e6086e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.518806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363752-771e-453a-81d4-8c683496d96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.519859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363752-771e-453a-81d4-8c683496d96a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.520373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363752-771e-453a-81d4-8c683496d96a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.520866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363752-771e-453a-81d4-8c683496d96a None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.521840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363752-771e-453a-81d4-8c683496d96a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:03.522351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363752-771e-453a-81d4-8c683496d96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:03.524074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363752-771e-453a-81d4-8c683496d96a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:03.534107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 407/786] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:03.539137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363752-771e-453a-81d4-8c683496d96a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.541415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.590484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52890b-c082-454a-b51c-ef8a54e78170 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.658950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.658950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.658950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.658950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.664709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363752-771e-453a-81d4-8c683496d96a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.685883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:03.697863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:03.698853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:03.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:03.703490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 380/787] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:03.704149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.705038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.705398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.705752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.711875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.717997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:03.719848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:03.749077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:03.750050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:03.750961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:03.750961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:03.750961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:03.750961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:03.759477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 408/788] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:03.760333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.768958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.850954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b648-5865-4992-840e-a8065ed8b593 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:03.875955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:03.889912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:03.890600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:03.890882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:03.891192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:03.892785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:03.892785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:03.893732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:03.905078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 381/789] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:03.911925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:03.958274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf529e-ac85-45e3-a5ac-8af5597139ce None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.023755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.024440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.024731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.025073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.025811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:04.026708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.028265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.058276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 409/790] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:04.063381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.108693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d6a3-4eeb-40bd-a687-2cfb4c77a4cf None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.176904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.177997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.178491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.179106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.184036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:04.193779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.197351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:04.233170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:04.234157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:04.235072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:04.235072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:04.235072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:04.235072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:04.239697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 382/791] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:04.243111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.278117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd6a69-7b07-4000-8cad-de9bb0faeabf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.323565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:04.352254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.353061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.353351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.353639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.368945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.370974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:04.398987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:04.400922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:04.402967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 410/792] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:04.411415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.430201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420fd22-2914-46fa-91bd-8274c24904f9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.497230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.497813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.498100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.498378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.499127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:04.499438 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.500809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.525249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 383/793] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:04.532459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.617680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546e13-1427-41b3-bd98-840040fe235d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.660241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:04.685376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.686055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.686337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.686863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.687742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:04.688056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.689486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.721316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 411/794] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:04.724313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adbf10-17ef-44d5-8f2c-d8f964923cd8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.726051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:04.726560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.789205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.789921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.790206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.790480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.804474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.805921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:04.833926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:04.836187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:04.838418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 384/795] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:04.843548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:04.915379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e0b-a746-4538-8e5a-8aadc7f8c773 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:04.975094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:04.975921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:04.976205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:04.976479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:04.992246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:04.993728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:05.027800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:05.028755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:05.029655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:05.029655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:05.029655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:05.029655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:05.031451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 412/796] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:05.034352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1642a6-4691-46bd-bb9d-901fe234fbcd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.036651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.073571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.074307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.074822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.075325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.075799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.077528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:05.078045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.079722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712d7de-8ef8-4793-bbd2-db253c8ddc99 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.102738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 385/797] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:26:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:05.104958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e64670-8598-437c-abad-82416c9a5d6f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.105434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e64670-8598-437c-abad-82416c9a5d6f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.105868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e64670-8598-437c-abad-82416c9a5d6f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.107581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 386/798] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:05.108800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e64670-8598-437c-abad-82416c9a5d6f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.136960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.153494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e52e-60ef-44e1-ab3e-c77866ac0f17 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.191952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.193376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.193938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.194394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.203827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:05.204148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.205540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f097eb-1fb4-4c3f-8355-35dd12d5499d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.218813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 413/799] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:05.220702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042425-2345-491d-8042-93c49910337b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.220702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042425-2345-491d-8042-93c49910337b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.221016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042425-2345-491d-8042-93c49910337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.246131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e64670-8598-437c-abad-82416c9a5d6f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.246476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-81042425-2345-491d-8042-93c49910337b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:05.247786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-81042425-2345-491d-8042-93c49910337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:05.288860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.289958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.290494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.291048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.339495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 414/800] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:05.339495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81042425-2345-491d-8042-93c49910337b None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.368265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.371021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.372486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc3b4b-1916-419e-83a3-5fe4a18826af None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.394510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.452484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81042425-2345-491d-8042-93c49910337b None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.478436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 387/801] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:05.480753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a35bc-df06-44c5-a0fb-476866d33078 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.481617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a35bc-df06-44c5-a0fb-476866d33078 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.482359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a35bc-df06-44c5-a0fb-476866d33078 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.484324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 388/802] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:05.485511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a35bc-df06-44c5-a0fb-476866d33078 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.489351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.490149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.490437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.490715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.497741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.499219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:05.513234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:05.514120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:05.515036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 415/803] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:05.515536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.622897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a35bc-df06-44c5-a0fb-476866d33078 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.626471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.642910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a6964-5448-4856-855c-b3e8683740e0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.645519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.659973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.660724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.661010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.661283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.671801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f4d319d2-6a9f-4ee6-b1f6-0eab8c06d3e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:05.680053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:05.681025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:05.681902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 389/804] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:05.683727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.683727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.683727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.692940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.693808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.694092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.694365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.702489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:05.703693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:05.706609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.708082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:05.725428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:05.726385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 416/805] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:05.727957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.790951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 390/806] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:05.790951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.831306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d4ab9-c21b-4c89-959f-3c77a555af29 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.863507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.864163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.864447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.864722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.865506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:05.865814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.867251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.870170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.890517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 417/807] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:05.890517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:05.897264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d01c-1a37-4497-8b1f-c0040d22b75e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:05.909764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:05.934201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:05.935715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:05.935715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:05.935715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:05.937237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:05.937237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:05.938344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:05.954198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 391/808] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:05.955059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.002675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0be2-d7d5-4c3f-8430-d7b076947a95 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.035515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.036176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.036458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.036736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.037448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:06.037756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.039137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ff346-1540-428a-98ed-6ed123e46f5f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.050396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 418/809] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:06.051700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150e9e-c53a-45f7-88d6-966e906843e2 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.051986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150e9e-c53a-45f7-88d6-966e906843e2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.052259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150e9e-c53a-45f7-88d6-966e906843e2 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.052752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f605-5440-4a8f-b8bf-a027969932a9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.053619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 419/810] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:06.054192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a150e9e-c53a-45f7-88d6-966e906843e2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.090031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.090834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.091150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.091431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.092087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.093458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:06.107742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:06.109367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.110363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 392/811] 10.222.0.36 () {56 vars in 1129 bytes} [Thu Jun 16 19:26:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:06.111285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.115951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.143848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.163909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a150e9e-c53a-45f7-88d6-966e906843e2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.196889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.197781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.198062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.198340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/e8e2bf373b15496597a6f18d14244495/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.200314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=e8e2bf373b15496597a6f18d14244495, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.201777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.212893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f73ed0-ae45-490d-8883-428853550449 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.246308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.247133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.247422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.247702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.259371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.260889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.261130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 420/812] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:06 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/e8e2bf373b15496597a6f18d14244495/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:06.262027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-20fb2ac4-90f3-4e73-8e26-f3ebd8ae868d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:06.282641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:06.283730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:06.284687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:06.284687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:06.284687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:06.284687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.284687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 393/813] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:06.285417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.285603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.286220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.304196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:06.305420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:06.377306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 394/814] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:06.378177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.391050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd36c8-8f0f-4691-8b50-39e3728e9702 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.400577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.410736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.438626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.439603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.440008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.440286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.448759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.450757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:06.466759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:06.467932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:06.469073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:06.469522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 421/815] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:06.470303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.497979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a351e0a-76da-40ee-a059-44de918b9486 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.534541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.535260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.535635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.535923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.536635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:06.536947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.538315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.543977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 395/816] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:06.544766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.577505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ef14-d915-40f9-b6f5-4e3ac57ac3af None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.611422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.612086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.612369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.612645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.613363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:06.613670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.615056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.626715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 422/817] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:06.631828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.632122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.652353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c99613-fd46-4873-8c3e-a2ff109c4043 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.665691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.689427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.690316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.690607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.690885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] PATH_INFO: `/v3/services/network` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.691572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.692997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:06.707609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:06.708873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 19:26:06.710197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:06.713843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 396/818] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:06.714692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.739019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ccb7-ea5b-47e0-a0d4-e1a1d380c7db None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.776293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.776960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.777248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.777526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.778311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:06.778626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.780077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.799638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 423/819] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:06.800372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.823938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54865-0e4a-4ac7-a80a-dc390475876f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.861670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.862509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.862806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.863113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.863823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:06.864131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.865611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.874370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 397/820] 10.222.0.36 () {56 vars in 1124 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:06.874370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:06.884088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.902776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:06.910047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1159ddb-92d9-4b29-b74e-ab45a7a53962 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.950546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:06.951394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:06.951681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:06.951957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:06.959605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:06.961047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:06.973988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02275f68-86f4-4cc0-91e1-25d7f5f8b60c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:06.979952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:06.981084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:06.982057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 424/821] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:06.982653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.012389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.013383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.013772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.014042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.014657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] RBAC: Adding query filter params (name=network) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:07.014990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.016407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.025343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 398/822] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:07.025343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.098375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f5c9-62cf-4a0a-924b-9c0dcf8e54e0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.101690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:07.126878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a85a2-1c19-4119-abbc-8e90a6d7c904 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.133249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.133908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.134195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.134466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.135423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:07.135743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.137170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.143159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 425/823] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:06 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:07.143754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.179979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:07.192901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.195093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.195483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.195861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.197461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.199662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.282377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0c434-99d8-40bc-9683-aaa968e92afa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.297884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 399/824] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:07.299296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.362806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.363667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.364002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.364283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.365203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:07.365521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.367321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.407293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 426/825] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:07.415376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.469930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c61ea-47ec-4aeb-b0a5-537d15c9fefe None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.518567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:07.524315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:07.538483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.539311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.539713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.540085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.540906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] RBAC: Adding query filter params (type=network) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:07.541308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.543407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.554352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 400/826] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:07.560413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.560552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acddc9-b5fd-404c-b07c-a19a4b537b1b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.603853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.604737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.605086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.605409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.623815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.625190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:07.646018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:07.647200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:07.648330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:07.648330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:07.648330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:07.648330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:07.648330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 427/827] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:07.653887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.695074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735883f-d9af-48bf-8f49-e4b7b4dcd6dc None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.754959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.755646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.755939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.756229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.757067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.758817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.759375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd63be6-b3e6-4b5c-8b3b-5a7397d9994a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:07.797863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:07.798575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:07.798862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:07.799170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:07.799974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:07.800285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:07.801687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:07.814279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 428/828] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:07.815299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:07.819822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:07.874203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 401/829] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:07.961988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2e8c-1270-4f0f-9e63-c2bb6e4dec11 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:08.005081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:08.005800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.006089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.006422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.022162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:08.023801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:08.043023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:08.044027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:08.045329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 429/830] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:08.050461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:08.121502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:08.203273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaab048-9899-43d2-a2f8-03494be8e302 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:08.240777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:08.241746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.242222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.242689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.243698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:08.244187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:08.246442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bed3e-8c23-4914-94c8-90c79a52478c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:08.261214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 402/831] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:08.267210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:08.312743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:08.371191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce7960-5162-49e6-ba27-b783eeb61a9b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:08.406042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:08.406963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.407241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.407516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/1e5c2a70586143ac94e5be00608aa6d8/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.423345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=1e5c2a70586143ac94e5be00608aa6d8, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:08.424992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e0d60-a314-4346-8987-99819183ddfc None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:08.535706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 430/832] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:08 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/1e5c2a70586143ac94e5be00608aa6d8/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:08.557974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df01dfa-daee-46ff-ba2e-574107008612 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.558309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df01dfa-daee-46ff-ba2e-574107008612 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.558624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df01dfa-daee-46ff-ba2e-574107008612 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.560421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 403/833] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:08.566540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.566907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.567224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.585876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:08.587170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:08.663310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 431/834] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:08.687819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df01dfa-daee-46ff-ba2e-574107008612 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:08.801404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df01dfa-daee-46ff-ba2e-574107008612 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:08.813054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:08.836966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:08.837997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:08.838993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:08.839445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:08.850012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:08.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:08.859561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:08.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:08.862975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 404/835] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:08.871184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:08.892001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:08.971826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf3f6f-d9c7-42d7-a479-f62df05be2a9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.006917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.007622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.007909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.008191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.008945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:09.009256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.011232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.020186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 432/836] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:09.025403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.128003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3163-d38f-4a73-b0f8-a068f09e146b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.163846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.163846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.163846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.163846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.176687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.178747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:09.200692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:09.201832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:09.202980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:09.202980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:09.202980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:09.202980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:09.204836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 405/837] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:09.210999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.297373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:09.314154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725715c2-23ca-46d3-a460-64717c38447d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.349011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.349992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.350433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.350892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.351818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:09.352277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.354315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.366867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 433/838] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:09.373132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.399228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:09.485238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93da41e-c997-4cf6-b3b0-df48549769cf None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.522234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.523096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.523385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.523666 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.532727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.534285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d508d4a6-c420-4797-ba7a-b7eda3472798 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:09.550765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:09.551824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:09.552787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 406/839] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:09.559662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.578860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cf56-f311-4175-b45d-301fd4f09345 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.579178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cf56-f311-4175-b45d-301fd4f09345 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.579454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cf56-f311-4175-b45d-301fd4f09345 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.581847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 407/840] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:09.586135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.586435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.586715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.606204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:09.607775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:09.685044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0f55-bffa-4e89-b48f-aec262d212d8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.695213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 408/841] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:09.717558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.724651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.725619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.726125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.726612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.727866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:09.728452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.731000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.733986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:09.751971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 434/842] 10.222.0.36 () {56 vars in 1135 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:09.758101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.847317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:09.855712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d6832-783f-4bc4-b7ef-fd86a7d9cd89 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.891252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360f47c-ac70-4e1e-90a3-e8d208ccc231 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:09.895760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.896575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.896860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.897138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.897799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:09.899213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b46f9103-fd22-498e-960b-0be3d78236e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:09.911956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:09.912906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:09.913750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 409/843] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:09.915553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942aab-173f-40a8-8895-de70b82ff61a None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.915553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942aab-173f-40a8-8895-de70b82ff61a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.915553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942aab-173f-40a8-8895-de70b82ff61a None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:09.916866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 410/844] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:09.919591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f942aab-173f-40a8-8895-de70b82ff61a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:09.932954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:09.933612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:09.933937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:09.934232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.008938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.015222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.019876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.019876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.034630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f942aab-173f-40a8-8895-de70b82ff61a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.073872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.074884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.075346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.075775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.076650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:10.077131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.078724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04cbbf-7f01-4b7c-9da2-337c6b342dfd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.084643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 411/845] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:10.086245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.086672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.087113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.105974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:10.107485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:10.187453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 412/846] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:10.188310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.250847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 435/847] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:10.252172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.261029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.293382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b641b-a028-4591-a837-c8b8910f2bb7 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.334426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.335492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.335776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.336202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.336902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:10.337212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.338647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.354961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 413/848] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:10.354961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.367551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa143-ecb9-4d71-86b1-ed145a18a5c9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.370107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.408332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.409266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.409626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.409899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.410663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.412135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:10.425711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:10.426770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:10.427851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 436/849] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:10.433602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.479774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e1b0-3e6b-4cd2-9fcd-33a279dfdce8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.529137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.530525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.531260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.532622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.535391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.548318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.550008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.570752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8034e-35ba-442d-9533-f5fb817375c9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:10.571246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:10.572113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:10.573063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:10.573063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:10.573063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:10.573063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:10.574686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 414/850] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:10.578789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.609001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.609452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.609739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.610018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.610750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:10.611085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.612452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.617446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 437/851] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:10.625884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.662506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.706145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4b179-e7ff-4832-bf78-a3523819d1ec None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.754915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.755774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.756133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.756486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.756597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a50d-ba2d-48b1-b6f1-76ee3d7e8cb5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.757350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:10.757731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.759386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fbfc6-5625-4380-9c07-497bdf7c160c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.773925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 415/852] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:10.778350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d30c-7ca0-44bf-a57a-4713a7fa540f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.778650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d30c-7ca0-44bf-a57a-4713a7fa540f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.779059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d30c-7ca0-44bf-a57a-4713a7fa540f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.780446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 416/853] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:10.784093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587d30c-7ca0-44bf-a57a-4713a7fa540f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.798569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.799267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.799562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.799969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.801044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:10.801356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.803155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb90f-d53c-4767-b872-d379e981c182 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.815072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 438/854] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:10.815072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.815072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.815072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.836216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:10.843976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:10.845815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:10.910054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587d30c-7ca0-44bf-a57a-4713a7fa540f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:10.943798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 439/855] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:10.958015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:10.958941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:10.959243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:10.959526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:10.967563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:10.968932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:10.970490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:10.987025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:10.988000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:10.988918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 417/856] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:10.994263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.000391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:11.092340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089535de-bdc5-4af7-a95d-c6a2985e0f64 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.130513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ab89-2a85-45fd-b47d-9e14a279f5fe None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.139213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:11.139662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.140459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.140761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.141014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.157204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:11.168385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:11.169322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:26:11.170156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:11.172682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 440/857] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:11.172986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.173726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.174082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.174429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.181071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:11.181071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.181071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.181343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.195754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 418/858] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:11.202840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.283124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:11.302468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06c2ba-a4bc-4df8-9ee7-248775d7bcd6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.323268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7236f-e1c9-42a3-96b2-09d54c943bd9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.345053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.346073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.346679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.347191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.348191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:11.348702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.350399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.358681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.359369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.359670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.359787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 441/859] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:11.359925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.360816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=e8e2bf373b15496597a6f18d14244495) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:11.361139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.362560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.370959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.371427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 419/860] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/role_assignments?user.id=e8e2bf373b15496597a6f18d14244495&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:11.494616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdee1a-97a2-4b50-9b45-25eb4b1ed5e3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.535383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.536187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.536470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.536748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.548310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.549768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.557033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:11.568103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:11.569090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:11.570031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:11.570031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:11.570031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:11.570031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:11.570031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 442/861] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:11.575720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.615376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:11.688949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77a75-31f6-4ea8-a619-7d45a33d608a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.724054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.724717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.725046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.725327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.726045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:11.726347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.727735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.739447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 420/862] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:11.745710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:11.857847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5abb-c4b4-4ca7-92bf-3e55a430c62b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:11.891745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:11.892528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:11.892866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:11.893144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:11.901560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:11.903009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:11.919079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:11.920059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:11.921175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 443/863] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:11.932149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.036403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.046361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32deedd-cc10-4e59-9263-c6725d8cccc6 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.081504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.082179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.082476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.082747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.083636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:12.083951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.085390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:12.108316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 421/864] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:12.109675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.174655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.221098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac5aeb-6a72-461e-a4b6-181066f0c6d3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.255551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.256208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.256490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.256767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.329248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.330695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5fbc0-05fd-43d8-9847-23efc5f8a359 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:12.389291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.432510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 444/865] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:12.439281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.548875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc2b14-88aa-42b2-b87f-b8ab0cab273a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.554430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13d53-0c79-45a3-8791-c845346c2f5e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.554723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13d53-0c79-45a3-8791-c845346c2f5e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.555027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13d53-0c79-45a3-8791-c845346c2f5e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.556625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 445/866] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:12.562488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.562776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.563086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.580144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:12.581329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:12.586555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.587387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.587674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.587954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.599064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.600523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:12.609984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:12.620863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:12.623172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:12.625038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:12.625038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:12.625038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:12.625038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:12.630957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 422/867] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:12.630957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.659184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 446/868] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:12.679432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.747600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a09fc-147e-40ea-bf19-abc37882130b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.792965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.794030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.795868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.796316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.797052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.798416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:12.799035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.801221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:12.807418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b15c-b5ac-45b3-9d08-33bf392167e1 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.822028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 423/869] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:12.823950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:12.852207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.852875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.853289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.854172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.854940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.857024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:12.893094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:12.931660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 447/870] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:12 2022] POST /identity/v3/services => generated 269 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:12.936045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f49ee-8e95-420d-98c5-f805743fb54d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:12.985367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:12.986560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:12.987071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:12.987538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:12.996187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:12.998087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-eda3fc28-0b02-4d3b-8573-f960219dcbf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:13.016448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:13.017431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:13.018347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:13.025655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 424/871] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:13.027428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:13.083308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fd2-f3a1-40dc-b7e6-6f2c2941a953 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.083598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fd2-f3a1-40dc-b7e6-6f2c2941a953 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.083881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06fd2-f3a1-40dc-b7e6-6f2c2941a953 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.085514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 425/872] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:13.091178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.091458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.091740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.109279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:13.110462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:13.153669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76705-0dac-483f-99a7-784f02e169f5 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:13.197703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:13.198377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.198666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.199142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.199870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:13.200194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:13.201671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:13.212023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 426/873] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:13 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:13.222944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 448/874] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:13.223113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:13.238697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:13.253135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:13.270625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:13.330071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b38a1f-35b1-48a0-a88f-4fdb7a04c9d3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:13.367366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f093c7-79fe-465f-923d-04b04519c2c2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:13.376349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:13.377711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.378192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.378630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.380330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=1e5c2a70586143ac94e5be00608aa6d8) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:13.380809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:13.383171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cd143-dadb-4b9f-afb0-f8f3f5793c50 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:13.399191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 427/875] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:13 2022] GET /identity/v3/role_assignments?user.id=1e5c2a70586143ac94e5be00608aa6d8&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:13.410817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:13.411550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.411834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.412109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.412851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:13.414266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cce23-3252-43b6-9d36-33c75a2a6c23 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:13.491842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 449/876] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:13 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:26:13.760779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a320dcf-147d-4bb5-833d-80ba19d20490 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.761309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a320dcf-147d-4bb5-833d-80ba19d20490 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.761619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a320dcf-147d-4bb5-833d-80ba19d20490 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.763420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 428/877] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:13.768864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:13.769274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:13.769657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:13.788248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:13.789455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:13.854291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:13.867776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 450/878] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:13.890196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a320dcf-147d-4bb5-833d-80ba19d20490 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:14.001589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a320dcf-147d-4bb5-833d-80ba19d20490 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:14.016617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:14.044913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:14.046368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:14.046842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:14.047432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:14.048631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:14.050869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:14.066953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:14.068053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:14.069263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 429/879] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:14.076986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:14.178483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ef6ad-4c96-4520-9bd9-cd4550084261 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:14.216099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:14.216778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:14.217063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:14.217341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:14.218050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:14.218357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:14.219741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479e71d-217b-40a1-bece-f14180f530f8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:14.229420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 451/880] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:14.232014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:14.343428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86e941-1329-492c-8d9c-32773aeae42a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:14.379072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:14.379738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:14.380021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:14.380295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:14.381003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:14.381310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:14.382677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729b330-aeca-42d5-83a7-935227845f64 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:14.388001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 430/881] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:15.174902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dbed-0cf7-4da4-833b-cf5b6380e04d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.175368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dbed-0cf7-4da4-833b-cf5b6380e04d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.175486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dbed-0cf7-4da4-833b-cf5b6380e04d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.177081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 452/882] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:15.184655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.184655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.184655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.203261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:15.204432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:15.235311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:15.274948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 431/883] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:15.297391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511dbed-0cf7-4da4-833b-cf5b6380e04d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:15.360547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:15.403087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511dbed-0cf7-4da4-833b-cf5b6380e04d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:15.439128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:15.439864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.440143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.440414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.441067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:15.442477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:15.454886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:15.455880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:15.456738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 453/884] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:15.464584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:15.569979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de845fc-4c0d-4606-9968-b3e2a1e2b8ca None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:15.605477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:15.606143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.606429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.606712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.607452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:15.607761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:15.609121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910c3da-6c66-4add-8995-6d7e4892cbdf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:15.614222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 432/885] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:15.619880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:15.718184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c9152-1046-48da-a4bd-9d9ca9d2bd68 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.718495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c9152-1046-48da-a4bd-9d9ca9d2bd68 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.718749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c9152-1046-48da-a4bd-9d9ca9d2bd68 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.720391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 433/886] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:15.725171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d2ab-7a4b-49a0-a24a-1672c954256d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.725460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d2ab-7a4b-49a0-a24a-1672c954256d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.725739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d2ab-7a4b-49a0-a24a-1672c954256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.731855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155e24-eefc-40de-a345-e7ad96841ab6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:15.742911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d7e0d2ab-7a4b-49a0-a24a-1672c954256d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:15.744151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d7e0d2ab-7a4b-49a0-a24a-1672c954256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:15.757533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:15.766134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:15.767001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.768753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.768753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.768753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:15.768753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:15.770425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ce830-692a-4f12-ab44-2b1f0a774574 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:15.771119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:15.781627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 454/887] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:15.782578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748f55-f3b6-4db6-b276-fad385bf3a70 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.782885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748f55-f3b6-4db6-b276-fad385bf3a70 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.783189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748f55-f3b6-4db6-b276-fad385bf3a70 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.784683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 455/888] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:15.785378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c748f55-f3b6-4db6-b276-fad385bf3a70 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:15.813482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 434/889] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:15.816260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.816552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.816830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.841100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:15.842616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:15.904084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c748f55-f3b6-4db6-b276-fad385bf3a70 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:15.911632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 435/890] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:15.912481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:15.952857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:15.953869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:15.954155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:15.954432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:15.955372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] RBAC: Adding query filter params (interface=public, service_id=6942e2ae4916458ebd826a303cbc6fed, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:15.955691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:15.957054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:15.969834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 456/891] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/endpoints?service_id=6942e2ae4916458ebd826a303cbc6fed&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:15.969834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.008744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.017802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0527f4-5619-4bee-bcab-c5aeba21bdb8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.026659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.063645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.064493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.064781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.065077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.076365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:16.080371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ca62-8941-4dfd-a068-56fa14613981 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:16.084747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:16.085687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:16.086516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:16.091243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 436/892] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:16.091964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.116762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.118070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.118552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.119098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.120077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.121745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:16.134410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:16.135985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:16.136811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:16.139070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 457/893] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:16.139950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.204609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a08be4-47a8-4bc2-a499-6d6be46fd788 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.245826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.246414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.246700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.247014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.247701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.249069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.254669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 437/894] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:16.255487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.277132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf4ef8-9245-40a0-8984-d36a46508104 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.283889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.303385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.320551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.322815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.323335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.324345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.326439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:16.327242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.329190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.342477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 458/895] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:16.347631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.395400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f5738-d446-4eab-91ad-22c963777e16 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.442868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.444993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.444993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.445412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.446279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:16.446749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.448900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.454281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 438/896] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:16.459633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.480565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae441-5b17-4c9b-8564-ce697fb3bcda None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.529351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.530495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.531045 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.531477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.543169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.544849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.558073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-95df8eb1-eca4-4bd6-b2c5-09fc73ad8d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:16.565785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:16.571506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 459/897] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730900b-d23a-4bd9-a9e4-ebe631b13cfa None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730900b-d23a-4bd9-a9e4-ebe631b13cfa None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730900b-d23a-4bd9-a9e4-ebe631b13cfa None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.572549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 460/898] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:16.574241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730900b-d23a-4bd9-a9e4-ebe631b13cfa None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.579067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa02216-85b9-491e-9b32-195ed000ab20 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.602758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.616304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.617347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.617347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.617636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.618299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:16.618607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.620637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95438915-95b6-466e-a9ed-409a4f2341c6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.630454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 439/899] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:16.630454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.630454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.630454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.646336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:16.647505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:16.689008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730900b-d23a-4bd9-a9e4-ebe631b13cfa None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.723077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 440/900] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:16.723077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.731284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.732429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.732722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.733008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.734240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:16.734565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.736269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.751096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 461/901] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:16.755838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.803605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.829242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f118a63-0152-4aaf-9ad2-13c484b1238f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.849998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:16.862708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.863397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.863553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d831c0-ce0b-4554-8174-38e2679b2789 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:16.863667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.863946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.864738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] RBAC: Adding query filter params (interface=public, service_id=e8af03caafa443ac9acbcfada327be6b, region_id=RegionOne) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:16.865050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.866493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.876671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 441/902] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/endpoints?service_id=e8af03caafa443ac9acbcfada327be6b&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:16.876671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.903708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:16.904383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:16.904670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:16.904949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:16.905671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:16.907297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:16.963927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 462/903] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:16 2022] POST /identity/v3/services => generated 241 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:16.964894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:16.975005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcf6b2-6100-4902-b8ab-f33a99642f07 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.021833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.023032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.023552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.023978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.027381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:17.034314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.036213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:17.054836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:17.056075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:17.057361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:17.059556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 442/904] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:17.065930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:17.087462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd888e-ad50-45ff-b404-8d8d55602800 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.127006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.127853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.128080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.128368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.129065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.130456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.138658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 463/905] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:16 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:17.199304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807198-9107-40ea-a765-33bb9db3fef6 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.239715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.241091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.241591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.242014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.243131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:17.243589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.245319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.265720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 443/906] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:17.271785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:17.293271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:17.381396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa8b6e-7ec5-408f-ba6f-720d00e9f6b9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.422717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.423657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.424086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.424519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.499262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.500713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.515098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:17.602451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 464/907] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:17.608213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:17.662692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:17.712089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1547e5-0ef9-488b-bb57-491da99db63a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.747861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.748657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.748945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.749223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.760514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.761967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:17.779372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:17.780318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:17.781280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:17.781280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:17.781280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:17.781280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:17.781280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 444/908] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:17.787618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:17.893145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe0d9-381c-4130-8250-168726f81425 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:17.907042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:17.928561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:17.929545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:17.930021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:17.930471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:17.931986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:17.932457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:17.934172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:17.946279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 465/909] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:17.952759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:18.054133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667938c-18e0-44ee-96f3-a578c7bbd750 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:18.057519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:18.097978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:18.098854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.099172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.099499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.107048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:18.108488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:18.123180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:18.124113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:18.125054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 445/910] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:18.130115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:18.239889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2123-43da-4f30-ad60-60e42525d239 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:18.277734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:18.278402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.278684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.278985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.279780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:18.280090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:18.281504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837a82-c137-4502-9fbe-699ed1350121 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:18.294868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 466/911] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:18.300696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:18.406174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3dca-05b5-470e-9ecb-4d343967604c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:18.442043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:18.442702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.443016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.447232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.447232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] RBAC: Adding query filter params (role.id=60a520a1a5264410a8cbc99fb1693f70, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=1e5c2a70586143ac94e5be00608aa6d8) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:18.447232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:18.447232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917b1b-5c4d-4535-a28d-f4a3d8982377 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:18.452025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 446/912] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:18 2022] GET /identity/v3/role_assignments?user.id=1e5c2a70586143ac94e5be00608aa6d8&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=60a520a1a5264410a8cbc99fb1693f70 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:18.552176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370830c-e588-433c-b31d-1cdc5ee76ccb None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.552361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370830c-e588-433c-b31d-1cdc5ee76ccb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.552647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370830c-e588-433c-b31d-1cdc5ee76ccb None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.554266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 467/913] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:18.559982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.560510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.560944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.578948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:18.580409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:18.607900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:18.655865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 447/914] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:18.679857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370830c-e588-433c-b31d-1cdc5ee76ccb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:18.713773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:18.791368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370830c-e588-433c-b31d-1cdc5ee76ccb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:18.827456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:18.828185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:18.828467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:18.828743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:18.829390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:18.830741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:18.843484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:18.844439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:18.845284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 468/915] 10.222.0.36 () {56 vars in 1132 bytes} [Thu Jun 16 19:26:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:18.852832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:18.962256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd2dce-446b-434d-be5b-b06d5e7f5c2b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.001808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.002835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.003346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.003821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.004748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:19.005263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.006817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.012847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 448/916] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:19.018893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.116154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.121683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6d27-4d81-47c7-a212-56692f46ee8a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.155849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.156518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.156799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.157075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.157780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:19.158165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.159553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeba45-ddf5-486c-87c1-7091337702d5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.164933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 469/917] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:19.172098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.213743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.257624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b460-ee04-4a97-974b-e30bf09b1ecd None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.258446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b460-ee04-4a97-974b-e30bf09b1ecd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.258963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b460-ee04-4a97-974b-e30bf09b1ecd None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.261926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 470/918] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:19.267764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.268342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.269224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.287921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:19.289655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:19.295563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5329c-4f5c-4875-ab41-561f6cb5df9d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.333959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.335335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.335771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.336198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.337238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.338908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e17114-f0a2-4b51-9947-ad1c6bb554cb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.362508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 471/919] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:19.387730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.477134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 449/920] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:19.478898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97810c-e98a-4142-9834-48ccffe3f874 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.479500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97810c-e98a-4142-9834-48ccffe3f874 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.479998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97810c-e98a-4142-9834-48ccffe3f874 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.481523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 450/921] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:19.487194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.487877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.488373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.494361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.507365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59c0a7-a105-4993-ae52-ae67ab64448a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.512312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:19.513963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:19.558725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.560018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.560550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.561306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.562448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.564192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:19.578120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:19.579352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:19.580292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:19.583543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 472/922] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:19.590355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.602999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 451/923] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:19.632899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.646575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.724423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f0676-5649-48c1-a308-9e1af5f52205 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.765687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a86c7-ee5c-40ec-8cab-6732001e8660 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.776207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.776917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.777195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.777482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.778196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:19.778514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.780406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.791234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 473/924] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:19.796440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.815278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.816087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.816377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.816809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.817422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.818261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.819739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:19.832352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:19.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:19.840625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 452/925] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:19.841420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:19.920852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0707d6-7969-4b53-8d23-fd53f4316f7a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.933574 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:19.957100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:19.958136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:19.958645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:19.959162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:19.960090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:19.960604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:19.962209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:19.968645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 474/926] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:19.971156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e42c-67f3-4d1e-b794-8c147610fe30 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:19.976563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:20.014443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.015639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.016417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.017163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.018050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:20.018553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:20.021162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:20.027756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 453/927] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:20.033412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:20.078520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:20.114406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae122b3-51e4-4c59-98f3-bf814366f9d1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:20.148544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fecc4-15b6-407a-bd9b-3b9596b8959a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:20.161299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.161987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.162529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.162821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.163674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] RBAC: Adding query filter params (interface=public, service_id=ab1de2dcf3aa4728b01f580f267e792b, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:20.163994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:20.166674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:20.179091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 475/928] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:19 2022] GET /identity/v3/endpoints?service_id=ab1de2dcf3aa4728b01f580f267e792b&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:20.180677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:20.194912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.195937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.196419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.196931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.197850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:20.198388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:20.200064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:20.202831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:20.211300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 454/929] 10.222.0.36 () {56 vars in 1114 bytes} [Thu Jun 16 19:26:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:20.217651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:20.322503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37a8ab-7e3b-4b57-9a95-1bf262975bec None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:20.326397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea806-3a3d-48fe-8a4b-6eb59ca0de95 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:20.366356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.367808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.368315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.369498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.370543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:20.371526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.372289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.372677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.372970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f4370-5d58-419c-9033-ead640d23b35 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:20.373134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.373983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:20.376187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e1476-4ca2-48b7-8b21-df72a113d13d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:20.380673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 476/930] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:20 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:20.491102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 455/931] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:20.721399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99ed9-d1f0-4cb4-8cdb-404c9bb7d93a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.721684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99ed9-d1f0-4cb4-8cdb-404c9bb7d93a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.721964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99ed9-d1f0-4cb4-8cdb-404c9bb7d93a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.723625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 477/932] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:20.729384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.729679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.729954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:20.746992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:20.748453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:20.804589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:20.828057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 456/933] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:20.852137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99ed9-d1f0-4cb4-8cdb-404c9bb7d93a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:20.863082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:20.961963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99ed9-d1f0-4cb4-8cdb-404c9bb7d93a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:20.997209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:20.997954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:20.998242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:20.998522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.009561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:21.017607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:21.018567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:21.019455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 478/934] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:21.024903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.133650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447d96d-a825-49e4-abdc-23c48526a8b4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:21.169336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:21.170150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.170434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.170716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.171564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:21.171871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:21.173243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:21.182641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 457/935] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:21.189709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.258525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:21.294909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce778de-945b-4061-b5cc-e6c5a0623e7e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:21.330033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:21.331251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.331753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.332226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.343361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:21.345115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:21.365894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:21.367438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:21.368379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:21.368379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:21.368379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:21.368379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:21.370812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 479/936] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:21.376679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.423357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:21.480617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edd79f-fc70-42d1-94d0-81a757c4fc26 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:21.516031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:21.517094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.517569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.518023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.518935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:21.528108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:21.528108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:21.533576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 458/937] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:21.540352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.645786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b3e8-146a-4d8d-aa82-32eb30600ff9 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:21.686796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:21.687723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.688197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:21.688694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.688980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.697882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:21.699404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-32d4e6e5-6dc6-4622-b520-b4e437123808 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:21.714569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:21.715624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:21.716655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 480/938] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:21.721794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.726954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe332c2e-7fab-492d-8332-621a03e58111 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.727245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe332c2e-7fab-492d-8332-621a03e58111 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.727525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe332c2e-7fab-492d-8332-621a03e58111 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.729143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 481/939] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:21.735720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.736004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.736625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.754478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:21.755848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:21.848400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597891-f642-4c1f-8be8-cb41beeea121 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:21.857932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 482/940] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:21.858082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:21.874515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:21.890061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:21.891370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:21.891917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:21.892529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:21.894149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:21.894752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:21.896823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:21.919130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 459/941] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:21.927845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.002871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5def02c-472e-4db2-a0e4-90c5935193d6 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.014750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.044841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.046792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.047546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.048064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.049246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.051393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.059170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70abe7c-38d4-4260-8198-9c24d7df0351 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:22.067626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:22.068819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:22.069877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:22.073511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 483/942] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:22.078758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.098566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.099812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.099956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.100301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.112220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.113666 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:22.135873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:22.136842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:22.137759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:22.137759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:22.137759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:22.137759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:22.141935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 460/943] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:22.143448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.160038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.221938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25fac-582f-4c9d-8653-afb12215b77e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.265679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.267067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.267574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.268257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.269846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:22.270358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.273018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.278178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2091eb-9be3-4799-addd-0754b80ad884 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.281438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 484/944] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:22.286216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.315674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.316497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.316791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.317088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.317801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:22.318137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.319564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.324627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.336592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 461/945] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:22.338469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.394968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26704e-06dd-487d-a0aa-ce05bb6a6ca2 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.425046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.435739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.436738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.437023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.441885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.441885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:22.441885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.441885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.442184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588c5c-b190-4635-b1ec-1d4ae653d486 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.444913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 485/946] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:22.480932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.481755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.483096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.483096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.490969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.492451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-4ed608fa-c1ca-4605-a9e4-392c4edb8844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:22.507508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:22.508509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:22.509463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:22.513755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 462/947] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:22.516099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.633151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.637075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c81f5-425a-4760-9cc5-654b66d04e6d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.668153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8612a4-d548-4310-a04e-16064af20959 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.668437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8612a4-d548-4310-a04e-16064af20959 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.668630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8612a4-d548-4310-a04e-16064af20959 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.673227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 463/948] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:22.676180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a02a43-676c-4af4-a995-bc43fb1cf3dc None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.676474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a02a43-676c-4af4-a995-bc43fb1cf3dc None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.676747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a02a43-676c-4af4-a995-bc43fb1cf3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.677128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.677801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.678085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.678364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.679262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:22.679575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.680995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.693788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 486/949] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:22.703037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.710321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a6a02a43-676c-4af4-a995-bc43fb1cf3dc None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:22.711640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a6a02a43-676c-4af4-a995-bc43fb1cf3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:22.807057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 464/950] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:22.813514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c3d4d-11e1-4f1f-8f4a-c59d954c9b1e None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.813802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c3d4d-11e1-4f1f-8f4a-c59d954c9b1e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.814084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c3d4d-11e1-4f1f-8f4a-c59d954c9b1e None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.817668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76607c-3891-4738-87af-00285ca9000c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:22.819149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 465/951] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:22.820675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.820967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.821250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.839360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:22.840549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:22.855911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:22.856803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:22.857088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:22.857371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/1e5c2a70586143ac94e5be00608aa6d8/roles/60a520a1a5264410a8cbc99fb1693f70` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:22.859889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=1e5c2a70586143ac94e5be00608aa6d8, role_id=60a520a1a5264410a8cbc99fb1693f70)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:22.861737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:22.899558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:22.908127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 466/952] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:22.911248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.951723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 487/953] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:22 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/1e5c2a70586143ac94e5be00608aa6d8/roles/60a520a1a5264410a8cbc99fb1693f70 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:26:22.951723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:22.981959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:23.033171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25af31-1f3b-44c4-9dcb-792db10cb8ef None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.084367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.085387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.086089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.086374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.087427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.089051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.095537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184a9dd-97b2-4bcf-8887-7de8d81ae580 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:23.105207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:23.106281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:23.107169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:23.107906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 467/954] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:23.113249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:23.142037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.143121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.143607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.144096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.144800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.146236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.147698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING keystone.server.flask.common [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 19:26:23.189717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:23.192283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.sql.core [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 19:26:23.192283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 19:26:23.192283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [parameters: {'id': 'c43e27aca8b6476c8d616ca7d34fce33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 19:26:23.192283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80308) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c43e27aca8b6476c8d616ca7d34fce33.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c43e27aca8b6476c8d616ca7d34fce33. Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:26:23.203210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 19:26:23.204011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application yield resource Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise self.value Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application session.commit() Jun 16 19:26:23.204976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.flush() Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application compat.raise_( Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 19:26:23.206040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 19:26:23.207154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:26:23.208112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [parameters: {'id': 'c43e27aca8b6476c8d616ca7d34fce33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.209185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:23.210099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c43e27aca8b6476c8d616ca7d34fce33. Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 488/955] 10.222.0.36 () {60 vars in 1140 bytes} [Thu Jun 16 19:26:22 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:23.211273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:23.251230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cf81-fd9d-4274-abac-3b188f2279e7 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.290153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.291400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.291685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.291966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.292762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:23.293072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.295104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.296580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:23.306763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 468/956] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:23.307983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:23.326673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4589117-6c6f-4a3e-adf0-34e0660dc099 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.361663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.362981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.363321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.363955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.372151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.373911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:23.388239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:23.389357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:23.390631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:23.396670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 489/957] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:23.397975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:23.427615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881f5dc-35e4-448b-9266-fce6e65e7274 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.454586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:23.467560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.468441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.468894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.469384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.470365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:23.470953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.472506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.483271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 469/958] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:23.484447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:23.502721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fc702-fb16-4f89-a2ce-4f9ffd9737b0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.537457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.538204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.538494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.538778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.539624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:23.540212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.542087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4223-ed3f-460e-a33b-18b06481da58 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.545966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:23.571080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 490/959] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:23.603917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf5349-6987-4860-b37c-93f7246f150e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:23.643414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:23.644095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:23.644378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:23.644657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:23.645418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:23.646811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f330bce-a981-40d7-9496-d6486a45110a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:23.730364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 470/960] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:24.661447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3630be-df00-4151-8667-7ea31a43dce5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:24.662005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3630be-df00-4151-8667-7ea31a43dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:24.662005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3630be-df00-4151-8667-7ea31a43dce5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:24.663617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 491/961] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:24.668997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8eb7-ec17-4e5c-8459-d81a6536fa41 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:24.669422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8eb7-ec17-4e5c-8459-d81a6536fa41 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:24.669599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8eb7-ec17-4e5c-8459-d81a6536fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:24.687487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d73d8eb7-ec17-4e5c-8459-d81a6536fa41 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:24.688688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d73d8eb7-ec17-4e5c-8459-d81a6536fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:24.726363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:24.767365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 471/962] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:24.791691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3630be-df00-4151-8667-7ea31a43dce5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:24.895777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3630be-df00-4151-8667-7ea31a43dce5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:24.930184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:24.930766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:24.931754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:24.932037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:24.932755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:24.934118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032349-a237-4ede-b9dc-105947c4e547 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:25.004957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 492/963] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:24 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:25.241731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6cdb-129a-4074-9021-d7eec499c0b3 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.242389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6cdb-129a-4074-9021-d7eec499c0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.242881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6cdb-129a-4074-9021-d7eec499c0b3 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.244823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 472/964] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:25.250141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.250666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.251196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.268795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:25.270656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:25.325979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:25.350328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 493/965] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:25.376785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d6cdb-129a-4074-9021-d7eec499c0b3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:25.492030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d6cdb-129a-4074-9021-d7eec499c0b3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:25.531232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:25.556465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:25.557060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:25.557940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:25.558852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 473/966] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:25.563789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:25.673184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6e2fe-73e0-4439-8855-f080aabccefa None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:25.708214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:25.708887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.709168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.709442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.710191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:25.710552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:25.711947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7d8e-01f7-46b8-b62f-525f01bc6df7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:25.720934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 494/967] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:25.726452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:25.758079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:25.840221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629249f-7221-4291-be4f-00874995fd42 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:25.850599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e702b6-9684-4085-b233-1d83b1d7c243 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.850915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e702b6-9684-4085-b233-1d83b1d7c243 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.851216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e702b6-9684-4085-b233-1d83b1d7c243 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.854596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 495/968] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:25.859315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8662b5-6319-4b53-9547-234bce5dfdf5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.859645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8662b5-6319-4b53-9547-234bce5dfdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.859874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8662b5-6319-4b53-9547-234bce5dfdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.878235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1a8662b5-6319-4b53-9547-234bce5dfdf5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:25.879515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1a8662b5-6319-4b53-9547-234bce5dfdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:25.881136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:25.882250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.883101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.883424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.897058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:25.899270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-032f6730-cd5c-4e87-9ed7-bf10e7bf616d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:25.921975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:25.922909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:25.923852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:25.923852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:25.923852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:25.923852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:25.924089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:25.925003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 474/969] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:25.929250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cf692-10bd-4d43-9dbb-2132db5da186 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.933794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cf692-10bd-4d43-9dbb-2132db5da186 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.933794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cf692-10bd-4d43-9dbb-2132db5da186 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.933794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 475/970] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:25.933794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cf692-10bd-4d43-9dbb-2132db5da186 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:25.962979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 496/971] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:25.963185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:25.963502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:25.963608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:25.980865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:25.982115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:26.020262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.041204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cf692-10bd-4d43-9dbb-2132db5da186 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.070950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 497/972] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:26.070950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.074810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.075549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.075846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.076128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.076861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:26.077176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.078547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.090980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 476/973] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:26.093238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.201094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1666-81d0-438d-81ee-ef95a9d7e00d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.204479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.208287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f2e6-6dac-4818-ac7c-f6c0aba634e8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.237727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.238407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.238691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.239006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.239714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.241202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.245875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.246716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.247032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.247312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.247972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.249323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:26.262078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:26.263345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:26.264245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 477/974] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:26.265603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.268838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.299644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.sql.core [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 19:26:26.299644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 19:26:26.299644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [parameters: {'user_id': 'e6a53a5198874dd58c57b8c254ef6a84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 19:26:26.299644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80308) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:26:26.307053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:26:26.314293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application yield resource Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application next(self.gen) Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise self.value Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application session.commit() Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 19:26:26.315528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.session.flush() Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application compat.raise_( Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 19:26:26.316463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 19:26:26.317350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application conn.query(q) Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application result.read() Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 19:26:26.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e6a53a5198874dd58c57b8c254ef6a84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:26.319060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 19:26:26.319906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 498/975] 10.222.0.36 () {60 vars in 1131 bytes} [Thu Jun 16 19:26:26 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:26.320851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.374037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb45b3-428c-46f6-88cb-51b782633888 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.409807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.410610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.410894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.411201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.419926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfcc1f-5a73-4860-a6fe-4563aa683b2d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.420725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.422160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:26.437294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:26.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 478/976] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:26.442600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.466487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.474791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.482969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 499/977] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:26.482969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.513984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.519751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.561487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af68261-b24b-4f50-9353-9e849e0ff1f2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.589230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a621cac-adab-4e7a-91b7-87a7a1f8f016 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.598467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.599292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.599576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.599851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.608301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.609672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:26.624432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:26.625499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 479/978] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:26.626479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.632958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.633644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.633930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.634239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.635119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:26.635435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.636859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.662665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 500/979] 10.222.0.36 () {56 vars in 1149 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:26.662665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.737134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7018115-4faf-4ffe-874c-6428f1b989ad None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.761858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.765252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:26.773098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.775338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.775338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.775338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.775636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:26.775875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.777252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.778123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf8280-69e4-4778-a9b3-8c70d13435f0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.787284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 480/980] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:26.787284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.819270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.819864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.820150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.820427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.821241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:26.821592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:26.823057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:26.859961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 501/981] 10.222.0.36 () {56 vars in 1101 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:26.861109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:26.912166 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc5b62-31d7-4ca6-9e30-3161414f5e5b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:26.954213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:26.954879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:26.955196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:26.955477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:26.991040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca323-0860-47ea-8c81-b5dff28dd8f4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.025628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.030906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.031629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.031914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.032191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.033010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] RBAC: Adding query filter params (interface=admin, service_id=ab1de2dcf3aa4728b01f580f267e792b, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:27.033326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.034721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.045696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 502/982] 10.222.0.36 () {56 vars in 1249 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/endpoints?service_id=ab1de2dcf3aa4728b01f580f267e792b&interface=admin®ion_id=RegionOne => generated 193 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:27.045890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.045890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.048671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.062352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.173340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332dae1-6596-4cce-b2a1-6e35c7c2b2a1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.188840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 481/983] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:26 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:27.194561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.214382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.215099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.215459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.215819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.216580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.218015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee8751-a957-4033-a44e-357092b4153c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.227090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 503/984] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:27.230195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bb8b4-8d49-4d69-b7df-1a14ca65a75e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.230491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bb8b4-8d49-4d69-b7df-1a14ca65a75e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.230775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bb8b4-8d49-4d69-b7df-1a14ca65a75e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.232222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 504/985] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:27.237913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.238198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.238484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.256194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:27.257764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:27.302338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.316625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce2e-b92c-43ab-a274-bb1e90df0d40 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.334383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 505/986] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:27.355558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.356367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.356653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.356921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.358146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.361927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.368276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.373564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:27.389433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:27.390393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:27.392771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:27.392771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:27.392771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:27.392771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:27.392771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 482/987] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:27.397299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.472960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0991d31-b17b-4a11-b28d-df866affe757 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.504346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7bc4-f40c-44a8-8f84-b85e659b3fae None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.508743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.510111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.510395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.510669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] PATH_INFO: `/v3/services/compute` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.511353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.512724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:27.524575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:27.525617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:27.526505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 506/988] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:27.531407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.543254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.543926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.544213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.544494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.545224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:27.545534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.547051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.555718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.560006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 483/989] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:27.566289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.606347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.641419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9fac-57e6-4bb2-947d-423d730779af None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.676267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1cd1a-42b8-49a0-b323-fa8382518ea0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.684051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.684733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.685018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.685300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.686269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:27.686582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.687977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.693103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 507/990] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:27.698980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.724161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.724885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.725175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.725459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.732425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.733939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:27.749839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:27.751246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:27.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 484/991] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:27.756920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.808962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e87c-f37c-44f8-a48c-22fb64716f8c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.820879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.842666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:27.847533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.848201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.848485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.848761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.849459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:27.849765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.851213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.856821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 508/992] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:27.863041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.866694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491cbbc-9aa6-4744-80e9-bc341920e768 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:27.902020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:27.902965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:27.903377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:27.903667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:27.904468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:27.904787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:27.906372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:27.923120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 485/993] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:27.927649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:27.968026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325737c-c373-4f66-884f-08b1681d7ffa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:28.005311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:28.005957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:28.006245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:28.006578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:28.007467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] RBAC: Adding query filter params (interface=public, service_id=baaad4dfdda246818eb24f75de06e009, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:28.007786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:28.009289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:28.019653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 509/994] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/endpoints?service_id=baaad4dfdda246818eb24f75de06e009&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:28.020611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:28.036973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f5a5f-0762-48bd-aabd-73ec8184c9f4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:28.057941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:28.074168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:28.075036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:28.075477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:28.085562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:28.085562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] RBAC: Adding query filter params (role.id=60a520a1a5264410a8cbc99fb1693f70, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=1e5c2a70586143ac94e5be00608aa6d8) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:28.085562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:28.085562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722d8e4-7ee5-4965-98b8-f81b9cc0798f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:28.085917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:28.086009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 486/995] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:27 2022] GET /identity/v3/role_assignments?user.id=1e5c2a70586143ac94e5be00608aa6d8&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=60a520a1a5264410a8cbc99fb1693f70 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:28.141083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b6a74-4236-4214-946a-116a145c1d22 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:28.184909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:28.186226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:28.186513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:28.186792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:28.187518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:28.188900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeda83-bfdf-41a2-8617-693540d11a95 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:28.196214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 510/996] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:28 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:29.380268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cf6b-cc65-4f1a-9947-94491e7fd9e7 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:29.380791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cf6b-cc65-4f1a-9947-94491e7fd9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:29.380896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cf6b-cc65-4f1a-9947-94491e7fd9e7 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:29.382226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 487/997] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:29.387516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:29.387914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:29.388103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:29.405162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:29.406353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:29.473922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 511/998] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:29.497638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745cf6b-cc65-4f1a-9947-94491e7fd9e7 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:29.596817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745cf6b-cc65-4f1a-9947-94491e7fd9e7 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:29.631478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:29.632274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:29.632560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:29.632836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:29.633488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:29.634883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:29.646785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:29.647811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 19:26:29.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:29.651237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 488/999] 10.222.0.36 () {56 vars in 1126 bytes} [Thu Jun 16 19:26:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:29.653838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:29.709964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:29.755383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df277c9-7847-4d58-a49d-8cac32aa80ed None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:29.791933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:29.792606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:29.792889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:29.793166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:29.793872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:29.794183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:29.795642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:29.803283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 512/1000] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:29.806131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:29.856183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:29.908401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4fd0-4599-41b1-bdc2-5bcb17a7ab70 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:29.942511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:29.943272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:29.943556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:29.943832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:29.944593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:29.944907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:29.946294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0dc21b-9307-421e-b4f6-b1245bbe21d5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:29.955242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 489/1001] 10.222.0.36 () {56 vars in 1122 bytes} [Thu Jun 16 19:26:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:29.956763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.057787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e4a2f-3435-4aca-a955-58f4ffdf4071 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.075851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:30.093414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.094080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.094365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.094643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.095418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.096777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602568b-53af-40cc-8293-3623fc5b5236 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.171745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 513/1002] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:26:30.283477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2bfe-0944-4452-804c-c2582bd8f941 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.283767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2bfe-0944-4452-804c-c2582bd8f941 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.284053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2bfe-0944-4452-804c-c2582bd8f941 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.285438 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 490/1003] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:30.291223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.291524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.291811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.309634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:30.311070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:30.326361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f706b33-e48a-447a-95e0-a92db4f6e6e7 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.326717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f706b33-e48a-447a-95e0-a92db4f6e6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.326946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f706b33-e48a-447a-95e0-a92db4f6e6e7 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.328521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 491/1004] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:30.333808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.334119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.334368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.355566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:30.356778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:30.395026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 514/1005] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:30.417640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.434034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 492/1006] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:30.457108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.508797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:30.529308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309bb04-7480-4f65-b5e3-321c588680d4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.531944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:30.566898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.567694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.567998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.568118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789daf1-abe8-489f-8198-28dd2ba5365a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.568293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.568928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.570301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:30.582523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:30.583927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:30.584989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:30.590968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 515/1007] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:30.594210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.611066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.611867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.612170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.612430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.613067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.614437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:30.626189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:30.627271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:30.628130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 493/1008] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:30.633071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.701586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb025ab-cb19-42de-a357-7583ef1273cf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.737255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295bf6-d415-4a2d-ab6e-056f8a33a75e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.738613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.739385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.739674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.739962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.740682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] RBAC: Adding query filter params (name=image) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:30.740997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.742363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.745594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:30.747834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 516/1009] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:30.753422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.779366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.780479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.781058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.781487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.782476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:30.783351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.785031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.796464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:30.797872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 494/1010] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:30.798624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:30.872037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe2a80-bce4-4001-856d-e726f93d296e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.908109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.909068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.909354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.909637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.910334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74165a-603a-4076-8161-8e96bb8cc242 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:30.910443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] RBAC: Adding query filter params (type=image) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:30.910651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.912061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.916918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 517/1011] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:30.956522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:30.957207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:30.957498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:30.957782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:30.958518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:30.958834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:30.960264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1597111-1ce3-4013-82f0-baf5c3039a48 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:30.966645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 495/1012] 10.222.0.36 () {56 vars in 1112 bytes} [Thu Jun 16 19:26:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:30.972839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:31.079911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:31.087816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b7a83-6a36-4ea4-a1a0-665e8cabf769 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:31.122118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:31.122709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:31.123021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:31.123300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:31.124049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:31.125429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42620590-bfcf-425b-b62d-ff3d48039865 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:31.204246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 518/1013] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:26:32.334225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db45-33a9-4295-b9e3-9a4c39f80e3a None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:32.334739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db45-33a9-4295-b9e3-9a4c39f80e3a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:32.334826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db45-33a9-4295-b9e3-9a4c39f80e3a None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:32.336421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 496/1014] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:32.342126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:32.342570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:32.342749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:32.359332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:32.360507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:32.434692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 519/1015] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:32 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:32.458147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154db45-33a9-4295-b9e3-9a4c39f80e3a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:32.494850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:32.563249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154db45-33a9-4295-b9e3-9a4c39f80e3a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:32.597242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:32.598040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:32.598327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:32.598608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:32.599815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:32.601238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506b662-265e-4a30-91ec-6bde5962a1a2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:32.607546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 497/1016] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:32.609033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:32.688467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:32.709990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53dddf-531c-464f-aead-42c8a20c5696 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:32.742850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:32.744257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:32.744543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:32.744869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:32.745764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:32.747291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa58cb-5097-4e03-abc5-c5dd37f76c92 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:32.850963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 520/1017] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:32 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:33.104909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143fd0d-5738-46c1-ab96-abd9e295aa9d None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.105325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143fd0d-5738-46c1-ab96-abd9e295aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.105509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143fd0d-5738-46c1-ab96-abd9e295aa9d None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.107013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 498/1018] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:33.112034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a419-07fe-4f4e-ad24-58a4a5d37932 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.112334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a419-07fe-4f4e-ad24-58a4a5d37932 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.112618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a419-07fe-4f4e-ad24-58a4a5d37932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.129823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3aa3a419-07fe-4f4e-ad24-58a4a5d37932 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:33.131034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3aa3a419-07fe-4f4e-ad24-58a4a5d37932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:33.202635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 521/1019] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:33 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:33.225450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143fd0d-5738-46c1-ab96-abd9e295aa9d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:33.328201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143fd0d-5738-46c1-ab96-abd9e295aa9d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:33.329731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:33.361886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:33.362601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.362839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.363104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.363841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:33.365227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:33.439326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f0aa-cb1e-40df-91f8-f349d92b8ac7 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.439852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f0aa-cb1e-40df-91f8-f349d92b8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.439940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f0aa-cb1e-40df-91f8-f349d92b8ac7 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.441306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 522/1020] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:33.445897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.446213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.446493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.463378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:33.464541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:33.477069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 499/1021] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:33 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:33.526279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:33.546495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 523/1022] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:33.569880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:33.678327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f553e-a8dd-431e-867e-d27a646a62c0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:33.711499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:33.712831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.713113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.713387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.723467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:33.732144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:33.733095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:33.734180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 500/1023] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:33.739178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:33.839218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839b3d7-9fbe-4c79-ac90-40a0ee33d5f3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:33.872401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:33.873071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:33.873358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:33.873637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:33.874386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:33.874694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:33.876081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:33.889046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 524/1024] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:33.890612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:33.916963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:33.994097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4958e2-9a9d-4272-b7ca-4d64f88ea7a4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:34.029541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:34.030335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:34.030625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:34.030904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:34.041153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:34.042606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:34.060010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:34.061198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:34.062111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:34.062111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:34.062111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:34.062111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:34.063249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 501/1025] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:34.067344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:34.117325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:34.172647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eec4e-0c46-4434-bd15-4aff5d90a984 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:34.209928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:34.210602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:34.210890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:34.211205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:34.211960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:34.212359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:34.213630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:34.225677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 525/1026] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:34.231453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:34.314032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:34.339023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab4a8b-896f-41af-8984-6e16c7a5e5fb None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:34.373879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:34.374666 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:34.374972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:34.375256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:34.383449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:34.384882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:34.398792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:34.400306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:34.401266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:34.403237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 502/1027] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:26:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:34.406536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:34.506502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2b9a2-5d27-4629-bc3b-dfe112df0935 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:34.516138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:34.541423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:34.542099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:34.542393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:34.542690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:34.543567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:34.544274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:34.546217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:34.568473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 526/1028] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:26:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:34.570002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:34.674634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee207b-4112-4d5f-b3b7-b54e3f7566c4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:34.708663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:34.709315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:34.709605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:34.709884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:34.712585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:34.779317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:34.780763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583ebf7-6032-4578-9276-efafd8f80f2a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:34.883384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 503/1029] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:34.885283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:34.990050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e181-b3bd-43b1-8c35-26aae7d71474 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.024961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.025757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.026042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.026318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.036261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d2c-af81-463c-8cb0-5f65b51385d7 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.036551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d2c-af81-463c-8cb0-5f65b51385d7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.036828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d2c-af81-463c-8cb0-5f65b51385d7 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.036931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.038321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 504/1030] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:35.038431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.043567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.043858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.044135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:35.058039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:35.058997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:35.059921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:35.059921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:35.059921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:35.059921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:35.061830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:35.063263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:35.063458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 527/1031] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:35.065053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.085335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.091954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.132528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 505/1032] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:35.157639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.181691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bff7-2b5e-41c5-8d6a-6f314e77b46d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.221912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.222747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.223076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.223682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.224695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:35.225049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.227166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.241161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 528/1033] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:35.248664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.292480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04058a-6dbb-427a-a62d-afd966bd391d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.339891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.340688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.340973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.341252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.342441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.343911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.350002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 506/1034] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:35.351018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.356112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.377315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67656b-7410-4d84-a7f6-f23be7f9cd76 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.381881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.417318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.418033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.418316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.418591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.425635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.427298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:35.443240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:35.445769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:35.447400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 529/1035] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:35.451145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.457115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e1e2-a390-4e5b-9d8b-4eaa8fedcf86 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.493670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.494342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.494628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.494906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.495661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.497092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff164e31-7edf-40a4-94fa-6fca436353bd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.590813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd3d2-31f0-41ed-a6a7-2f24042e6815 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.603348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 507/1036] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:26:35 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:26:35.615181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce965e2-c4ca-4a98-ae34-ccad05c535b4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.615597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce965e2-c4ca-4a98-ae34-ccad05c535b4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.615965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce965e2-c4ca-4a98-ae34-ccad05c535b4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.617649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 508/1037] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:26:35.626450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.626741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.627236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.649002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:35.650907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:35.652433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.653127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.653412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.653685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.654491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:35.654805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.656335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.664244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.670557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 530/1038] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:35.678637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.737261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 509/1039] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:35.761110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:35.780328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:35.796125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb48a2c-c0a2-4d9c-bedb-00ac4feb1fbf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.834366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.835429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.836258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.836258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.837002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] RBAC: Adding query filter params (role.id=60a520a1a5264410a8cbc99fb1693f70, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:35.837391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.838903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.845614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 531/1040] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=60a520a1a5264410a8cbc99fb1693f70 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:35.880403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679b01c-b36a-4289-b274-ac27d65c9d88 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:35.919744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:35.920962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:35.921448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:35.921895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] PATH_INFO: `/v3/services/image` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:35.922727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:35.924358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:35.938238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:35.939459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:35.940352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:35.941684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 510/1041] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:35.947133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:36.016268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:36.056467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c4d6-2a26-4089-a20b-87d6f584819a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:36.089649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:36.090328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:36.090619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:36.090901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:36.091636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] RBAC: Adding query filter params (name=image) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:36.091952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:36.093312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:36.103335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 532/1042] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:36.104501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:36.197609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:36.215667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86428576-bf1d-43b9-b840-018615b2252c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:36.249745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:36.250408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:36.250693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:36.251002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:36.251711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] RBAC: Adding query filter params (type=image) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:36.252015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:36.253373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:36.265847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 511/1043] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:36.267221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:36.375608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcfb66-4687-4e7f-ac16-f27286987329 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:36.401452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:36.410374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:36.411067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:36.411354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:36.423904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:36.423904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] RBAC: Adding query filter params (interface=public, service_id=d08008a55eca482d81c897fdf6c8ecff, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:36.423904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:36.423904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76e36b-a91a-4881-b566-06b0088775c9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:36.423904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 533/1044] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:26:36 2022] GET /identity/v3/endpoints?service_id=d08008a55eca482d81c897fdf6c8ecff&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:36.425079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:36.525982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9c7f-7ef3-4975-9f14-ab67996e754a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:36.559544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:36.560211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:36.560497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:36.561027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:36.561710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:36.563098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041e84d-fd4b-4dc1-8728-473152d74372 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:36.575157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 512/1045] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:26:36 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:26:37.816843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f5d6-cd48-4c7e-a1a3-178bf990a4b3 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:37.817403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f5d6-cd48-4c7e-a1a3-178bf990a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:37.817403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f5d6-cd48-4c7e-a1a3-178bf990a4b3 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:37.818818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 534/1046] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:37.824434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c1dc0-28b0-4353-a835-c7b24d62ec52 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:37.824847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c1dc0-28b0-4353-a835-c7b24d62ec52 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:37.825020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c1dc0-28b0-4353-a835-c7b24d62ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:37.842172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fc2c1dc0-28b0-4353-a835-c7b24d62ec52 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:37.843409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fc2c1dc0-28b0-4353-a835-c7b24d62ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:37.846211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:37.916420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 513/1047] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:37.940612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f5d6-cd48-4c7e-a1a3-178bf990a4b3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:37.965344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40c61b-6a2d-4e42-89f6-98b5fa97cdbd None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:37.965690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40c61b-6a2d-4e42-89f6-98b5fa97cdbd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:37.965916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40c61b-6a2d-4e42-89f6-98b5fa97cdbd None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:37.967297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 514/1048] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:37.972248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:37.972651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:37.972930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:37.990868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:37.992292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:38.043487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f5d6-cd48-4c7e-a1a3-178bf990a4b3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.063595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 515/1049] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:38.081712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.082440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.082719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.083020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.086875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.094562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:38.102031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:38.103034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:38.103900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 535/1050] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:38.109128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.163843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.164040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.202302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9deaf-c96f-4f07-b4f8-fd78770c9318 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.222569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688177c-f1c9-4a4f-b8ff-c3be95de3239 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.239937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.240757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.241047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.241323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.252579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:38.260598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.261273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.261561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:38.261679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:38.262568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:38.263730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:38.264821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.264821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:38.264821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.265041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 516/1051] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:38.268360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.268882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.279307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 536/1052] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:38.283319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.390998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbce700-2b70-44af-80ee-3d5ac248b434 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.397880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b82e2-0fc1-4baa-b4b3-52bbca1f7e9d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.410530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.421180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.429077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.429857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.430807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.431150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.431956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:38.432272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.433750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.433978 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.434876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.435363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.435906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] PATH_INFO: `/v3/users/27e367f21382486583cef03f71d5743d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.444086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 517/1053] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:38.449262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.454122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] RBAC: Authorizing `identity:get_user(user_id=27e367f21382486583cef03f71d5743d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.455611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.462171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 537/1054] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/users/27e367f21382486583cef03f71d5743d => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:38.463917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.559911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae185eb-2d11-4725-99ec-0b0d3a0514e0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.563524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266f63-25a2-487f-bb73-f90d87e9718c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.597481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.598134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.598416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.598620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.598766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.599450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.599739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.600022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.611460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.612915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:38.631376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:38.632527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:38.633607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:38.633607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:38.633607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:38.633607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:38.633607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 518/1055] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:38.638707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.644529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.676685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.678914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad5334-1e97-4dc2-a09c-6499bb557105 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.689378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.743950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46691a2-fb27-4286-b783-4e31bb75ae61 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.776614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.777273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.777559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.777841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.778433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 538/1056] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 19:26:38.778563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:38.778868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.779790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228339e2-7261-428a-ae25-0711f2925d62 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.780081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228339e2-7261-428a-ae25-0711f2925d62 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.780362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228339e2-7261-428a-ae25-0711f2925d62 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.780473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f7c46-fa4b-41c6-ae80-db9efb867ad1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.781780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 539/1057] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:38.786219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228339e2-7261-428a-ae25-0711f2925d62 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.794243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 519/1058] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:38.795633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.795924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.796201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.812178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:38.813358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:38.855608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.878484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 520/1059] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:38.879419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.891982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228339e2-7261-428a-ae25-0711f2925d62 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:38.913619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:38.928293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:38.929153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:38.929440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:38.929714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:38.931053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:38.932478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:38.934188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 540/1060] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:38.934964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:38.980918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004d1f-8032-4949-b585-32aec3a2d7c4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.014601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.015415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.015693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.015975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.023124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.025102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.032883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13cd04-286a-4808-9f5d-05e6bd81e75e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:39.039338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:39.040968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 521/1061] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:39.041935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.066620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.067450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.067982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.067982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] PATH_INFO: `/v3/services/image` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.068642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.069987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:39.081722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:39.082806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:39.083737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 541/1062] 10.222.0.36 () {56 vars in 1105 bytes} [Thu Jun 16 19:26:38 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:39.084561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.086332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.140300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.147692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bff21-1bdc-425d-8029-edb6fb869ac8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.181972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.182633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.182947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.183218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4aa22-db58-49c2-b1ef-76e935820e2e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.183325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.184107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1, user.id=27e367f21382486583cef03f71d5743d) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:39.184427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.186142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.191734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 522/1063] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/role_assignments?user.id=27e367f21382486583cef03f71d5743d&scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:39.192525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.233851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.234537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.234818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.235122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.235977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:39.236291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.237736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.257909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 542/1064] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:39.263774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.307746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160f3b-0aa9-4b61-b431-b947b8958432 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.332634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.346688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] RBAC: Adding query filter params (name=image) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.352535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.356414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 523/1065] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:39.361941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.384171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5b403-28ff-4906-a30e-403a5460ca49 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.389992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.421649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.422378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.422664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.422968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.435166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.436871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:39.454882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:39.455843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:39.461963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:39.461963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:39.461963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:39.461963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:39.461963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 543/1066] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:39.462244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.476784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19ae54-a08f-41f0-aba2-7d45ba925b65 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.514360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.515056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.515343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.515619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.516418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:39.516730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.518098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.523383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 524/1067] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:39.529634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.571232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25617020-24a4-415d-b41b-761eb6acd43c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.588378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.611366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.612042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.612325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.612602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.613332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:39.613642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.615044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.626743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 544/1068] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:39.632170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.633054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:39.650268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb308-682a-4637-9fc1-30458e4fbed7 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.688312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.689002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.689292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.689576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.690320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.692012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.744162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee662d-6e75-40ed-8949-4c4da1fc7026 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:39.774970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 525/1069] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:26:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:26:39.804896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:39.806238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:39.806828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:39.807183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:39.815909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:39.817370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:39.839930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:39.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:39.842553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 545/1070] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:39.847055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:39.981597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ee8be-8c26-41c0-8226-abcd4a7c4807 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:40.018190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:40.019869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:40.020164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:40.020449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:40.021253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:40.021709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:40.023144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53696fa9-4783-4b15-9a66-1a787b69ea37 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:40.036384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 526/1071] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:40.043638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:40.073958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:40.152890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858a474-3012-4731-badc-152c0fbb3657 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:40.194498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:40.195438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:40.195729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:40.196012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/60a520a1a5264410a8cbc99fb1693f70` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:40.198296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=a82e215bee54480b8e4f69ca6352a865, role_id=60a520a1a5264410a8cbc99fb1693f70)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:40.199775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d5951-0fd5-456a-9300-d852ca3292f4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:40.271935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 546/1072] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:40 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/a82e215bee54480b8e4f69ca6352a865/roles/60a520a1a5264410a8cbc99fb1693f70 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:41.312067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00758-7d78-47bb-9d44-491454b542f0 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.312621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00758-7d78-47bb-9d44-491454b542f0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.312621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00758-7d78-47bb-9d44-491454b542f0 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.314015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 527/1073] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:41.319159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.319545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.319723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.337674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:41.338858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:41.409692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 547/1074] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:41.433253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a00758-7d78-47bb-9d44-491454b542f0 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:41.436419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:41.538599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a00758-7d78-47bb-9d44-491454b542f0 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:41.572501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:41.573292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.573577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.573850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.583949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:41.592115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:41.593089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:41.593963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 528/1075] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:41.601314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:41.620535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:41.702886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e3081-e7b3-4b63-b8d1-02e15e6f99b0 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:41.736339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:41.737018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.737309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.737588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.738338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:41.738649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:41.740036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:41.749880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 548/1076] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:41.754306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:41.818407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:41.857029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd105-1c6a-4458-b7bb-9b0cb3a2715b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:41.891317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:41.892223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.892507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.892786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] PATH_INFO: `/v3/users/27e367f21382486583cef03f71d5743d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.894115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=27e367f21382486583cef03f71d5743d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:41.895624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55a6a1-5708-4d17-996d-2c5aaff94ec8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:41.897541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 529/1077] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:41 2022] GET /identity/v3/users/27e367f21382486583cef03f71d5743d => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:41.903346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:41.959750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5829f-2dfc-4f91-9112-859a1a10f680 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.960050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5829f-2dfc-4f91-9112-859a1a10f680 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.960235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5829f-2dfc-4f91-9112-859a1a10f680 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.961738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 530/1078] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:41.966599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:41.966893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:41.967206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:41.984527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:41.985822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:41.987685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.016162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70957d31-0f3b-4ebb-9b17-85542a4c5a2e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.059775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.060906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.061190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.061464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.062623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.064074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.065771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 549/1079] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:41 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:42.067033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 531/1080] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:42.076016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.089188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.146993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.189366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d456f3e-21b8-437a-8e10-6a230293099e None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.197996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801610-e782-4d22-ba6b-332b54cbeca0 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.225790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.227390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.227680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.227959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1/users/27e367f21382486583cef03f71d5743d/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.230057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] RBAC: Authorizing `identity:create_grant(project_id=c69290eda4d94f11a4db3b7566a3ccc1, user_id=27e367f21382486583cef03f71d5743d, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.231717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.231822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.232367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.232654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.232928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.298426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.299926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87345a9-3267-4db1-87b9-9dd8e5d778a6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.323335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 550/1081] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:42 2022] PUT /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1/users/27e367f21382486583cef03f71d5743d/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:42.362992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.415410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 532/1082] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 19:26:42.421780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.435136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.446771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817700dd-a455-4c0c-905a-ade54eb3d9b9 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.447098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817700dd-a455-4c0c-905a-ade54eb3d9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.447390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817700dd-a455-4c0c-905a-ade54eb3d9b9 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.448814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 533/1083] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:42.454615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.454966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.455201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.476823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:42.478913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:42.542104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed499-a59b-4a73-aeeb-cbba44253084 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.554130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 534/1084] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:42.576877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.577816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.577950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.578241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.578528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.590250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.591724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:42.609737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:42.610684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:42.611640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:42.611640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:42.611640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:42.611640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:42.611640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 551/1085] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:42.616743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.675593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.678108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:42.688960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2f2d-d0d2-4a3e-91ce-02422d0d7142 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.717655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa5ef-71dc-43f8-8729-47a7b3fdc7e0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.722952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.723778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.724067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.724375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.736228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:42.745699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:42.746705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 19:26:42.747578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:42.750352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 535/1086] 10.222.0.36 () {56 vars in 1120 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:42.753724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.754423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.755001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.755104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.755753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:42.755867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:42.756068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.757434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.778957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 552/1087] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:42.879992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ca87-c915-4129-8ecc-4906ff96d076 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:42.916364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:42.917047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:42.917338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:42.917637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:42.918389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:42.918702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:42.920128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:42.929167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 536/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:42.935744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:43.048746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03671f-f5fa-4648-a310-e8cdc63135e4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:43.069439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:43.085118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:43.085921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:43.086768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:43.087082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:43.097090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:43.098523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:43.115351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:43.116262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:43.117266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:43.117266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:43.117266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:43.117266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:43.117266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 553/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:43.122303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:43.137792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:43.220199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5be398-6807-48f6-a83a-b10e6b54aac7 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:43.252578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:43.253230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:43.253518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:43.253797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:43.254512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:43.254818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:43.256211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:43.267570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 537/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:43.273096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:43.388277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe7bf-7ff9-4f71-9f3e-301e495faefb None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:43.421732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:43.422528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:43.422813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:43.423119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:43.431021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:43.432445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:26:43.447414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:43.448375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:26:43.449349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 554/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Thu Jun 16 19:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:43.454397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:43.522005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:43.550733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8af67-10e4-4064-9c68-fa4634c4c590 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:43.583366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:43.584017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:43.584303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:43.584583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:43.585409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:43.585720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:43.587201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:43.603914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 538/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Thu Jun 16 19:26:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:43.609623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:43.640318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:43.718665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c137-54bb-4cd0-9459-e320d50b943d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:43.752487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:43.753120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:43.753403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:43.753679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:43.817348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:43.818785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:43.846419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:43.912239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 555/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:43.918347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.014191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9aff7-4cc0-408d-bfb3-40dabf1ce752 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.042136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:44.047459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.048300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.048582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.048860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.058423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.059883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:26:44.076973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:44.078021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:44.079094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:26:44.079094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:26:44.079094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:26:44.079094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:44.079094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 539/1094] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:44.083713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.184080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807adb3-07c4-49bc-a173-cfb7d497cc55 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.212862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:44.217701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.218365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.218659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.219050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.219682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:44.227377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.227377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.232763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 556/1095] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:44.238398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.333767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39185b-7444-40f3-a4cf-a52d38f6b6c1 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.366326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.367134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.367416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.367686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.374099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.375560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-1623b772-861f-4b7c-aefa-4ff4e41d88cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:26:44.391387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:44.393238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:44.395649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 540/1096] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:26:44.399774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.467050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ae1d-ec29-4e13-961b-a435770a5584 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.467586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ae1d-ec29-4e13-961b-a435770a5584 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.467586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ae1d-ec29-4e13-961b-a435770a5584 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.468950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 541/1097] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:44.473750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.474040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.474321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.491267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:44.492445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:44.501899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52317-0cbe-4871-9727-64f51d1c098a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.527475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:44.535452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.536167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.536452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.536733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.537530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:44.537844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.539301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.551019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 557/1098] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:44.556922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.558493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 542/1099] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:44.582019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.608585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:44.654426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf241-1e07-45ad-904d-2f6226fa2b65 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.678019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f068a-27a7-433f-a328-f974feb21226 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.687767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.688419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.688706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.688983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.689817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] RBAC: Adding query filter params (role.id=60a520a1a5264410a8cbc99fb1693f70, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=a82e215bee54480b8e4f69ca6352a865) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:44.690128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.691595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.697805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 558/1100] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/role_assignments?user.id=a82e215bee54480b8e4f69ca6352a865&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=60a520a1a5264410a8cbc99fb1693f70 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:26:44.713033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.713856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.714147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.714430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.726570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:26:44.734382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:26:44.735738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:26:44.736661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 543/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:26:44.742018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.848995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7675-6188-4c64-8d61-29c1917fc814 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:44.866358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:44.884607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:44.885342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:44.885629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:44.885907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:44.886654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:44.887030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:44.888395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:44.896695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 559/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:44.902769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:44.935793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:45.006659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9216a1-81c3-44c4-a59a-d2d4c739241d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:45.041912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:45.042780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:45.043112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:45.043388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] PATH_INFO: `/v3/users/27e367f21382486583cef03f71d5743d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:45.044578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] RBAC: Authorizing `identity:get_user(user_id=27e367f21382486583cef03f71d5743d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:45.046001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:45.047707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 544/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:44 2022] GET /identity/v3/users/27e367f21382486583cef03f71d5743d => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:45.053523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:45.154800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a3ec3-b8c2-4f37-b3ea-e0f7bf37e56a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:45.193146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:45.193828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:45.194112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:45.194393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:45.240081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:45.262327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:45.263744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:45.368646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 560/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 19:26:45.374625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:45.475495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:45.479004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9026-0503-4b41-ac89-edb894ae0f9c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:45.511763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:45.512670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:45.512899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:45.513180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:45.514365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:45.516101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a3569-8ed3-4f50-979e-d4e3556528bc None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:45.519078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 545/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:45 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:45.524607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:45.604807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:45.625412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba11f-1eff-41f9-9f28-47ad6d891ce6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:45.657975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:45.658622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:45.658905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:45.659209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:45.660031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1, user.id=27e367f21382486583cef03f71d5743d) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:45.660340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:45.661745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bd133-e485-4976-8391-1cabdeef52a4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:45.668044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 561/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:45 2022] GET /identity/v3/role_assignments?user.id=27e367f21382486583cef03f71d5743d&scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:47.776103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92658-be69-470d-b436-71b7894427d9 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:47.776637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92658-be69-470d-b436-71b7894427d9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:47.776637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92658-be69-470d-b436-71b7894427d9 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:47.778010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 546/1107] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:47.783235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:47.783664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:47.783795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:47.800767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:47.801955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:47.865931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 562/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:47.890165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f92658-be69-470d-b436-71b7894427d9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:47.988449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f92658-be69-470d-b436-71b7894427d9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:48.020141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:48.020911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:48.021196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:48.021475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:48.022630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:48.024622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f6ba5-5fad-4a03-aa46-c3d9ab2fbd56 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:48.026231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 547/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:48.031650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:48.099408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:48.126467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8c853-e7b7-425f-92ea-dd2e80889735 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:48.159625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:48.160278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:48.160560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:48.160836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:48.161536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:48.162977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910458b8-bfae-4b4d-95b5-266a9ae8401f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:48.229572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 563/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:26:48 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:26:50.363908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670d2aa-b86d-414a-90fe-1b20df2700c1 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:50.364490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670d2aa-b86d-414a-90fe-1b20df2700c1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:50.364490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670d2aa-b86d-414a-90fe-1b20df2700c1 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:50.365810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 548/1111] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:50.370832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:50.371307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:50.371405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:50.388760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:50.389953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:50.459120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 564/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:50.482729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670d2aa-b86d-414a-90fe-1b20df2700c1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:50.518478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:50.580695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670d2aa-b86d-414a-90fe-1b20df2700c1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:50.614028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:50.614864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:50.615183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:50.615455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] PATH_INFO: `/v3/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:50.617490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] RBAC: Authorizing `identity:get_role(role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:50.618852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:50.620218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 549/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:50 2022] GET /identity/v3/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:50.625785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:50.681598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:50.721069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3ea56-7627-4b8b-bfee-eab0a8b84217 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:50.771283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:50.772152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:50.772435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:50.772708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] PATH_INFO: `/v3/users/dff7f668779e4de38ab4ac828e5df2d5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:50.785732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] RBAC: Authorizing `identity:get_user(user_id=dff7f668779e4de38ab4ac828e5df2d5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:50.787190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:50.789479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 565/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:50 2022] GET /identity/v3/users/dff7f668779e4de38ab4ac828e5df2d5 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:50.795209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:50.889063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95e099-8536-4f98-8b1e-85db2b2f3839 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:50.891322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:50.922670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:50.923328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:50.923614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:50.923890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:50.987274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:50.988641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:51.081431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 550/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:51.088124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:51.182847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532017-c091-4583-9444-9c1f518a5b8c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:51.199114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:51.216501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:51.217354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:51.217638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:51.217912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:51.219112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:51.220527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b4aaa-f75a-430f-a30a-042b6af9c229 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:51.222246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 566/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:51 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:51.228247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:51.231293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:51.323095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be498f-4771-4af4-9cc3-80a99a34744f None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:51.354747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:51.355413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:51.355696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:51.355972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:51.356791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1, user.id=dff7f668779e4de38ab4ac828e5df2d5) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:51.357100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:51.358512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076548-5d65-4eb0-bbb6-6d6b423c3b8a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:51.363745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 551/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:51 2022] GET /identity/v3/role_assignments?user.id=dff7f668779e4de38ab4ac828e5df2d5&scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:53.437842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578a24a-54d6-45f6-a8af-8407818e7f28 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:53.438355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578a24a-54d6-45f6-a8af-8407818e7f28 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:53.438355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578a24a-54d6-45f6-a8af-8407818e7f28 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:53.439887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 567/1118] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:53.444903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:53.445326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:53.445455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:53.461596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:53.462762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:53.521477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:53.526564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 552/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:53.549052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578a24a-54d6-45f6-a8af-8407818e7f28 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:53.646904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578a24a-54d6-45f6-a8af-8407818e7f28 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:53.678598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:53.680059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:53.680340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:53.680611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] PATH_INFO: `/v3/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:53.682067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] RBAC: Authorizing `identity:get_role(role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:53.683467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:53.684812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 568/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:53 2022] GET /identity/v3/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:53.690776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:53.785006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cf7f-dc02-4f41-aa7f-1dc8c2e91f87 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:53.817551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:53.818402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:53.818680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:53.818976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] PATH_INFO: `/v3/users/dff7f668779e4de38ab4ac828e5df2d5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:53.820161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] RBAC: Authorizing `identity:get_user(user_id=dff7f668779e4de38ab4ac828e5df2d5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:53.821578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:53.823201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 553/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:53 2022] GET /identity/v3/users/dff7f668779e4de38ab4ac828e5df2d5 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:53.828754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:53.880160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:53.925304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb08c8-0608-4f65-b83a-b2f0063a6cff None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:53.957059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:53.957851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:53.958135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:53.958418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:53.959590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:53.961486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f52ec-c57b-4765-8039-956974b858a0 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:53.963180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 569/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:53 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:53.968804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:54.032201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:54.066120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191c33-eb8b-4773-bc19-8e9ab905495e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:54.097854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:54.098718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:54.099025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:54.099302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1/users/dff7f668779e4de38ab4ac828e5df2d5/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:54.101968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c69290eda4d94f11a4db3b7566a3ccc1, user_id=dff7f668779e4de38ab4ac828e5df2d5, role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:54.103419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8458363-59b4-4b85-becb-dc28f6e7faf8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:54.172068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 554/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:26:53 2022] PUT /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1/users/dff7f668779e4de38ab4ac828e5df2d5/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:26:56.239672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e8bb7-b973-4cad-b6eb-3064d4413b7c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:56.240224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e8bb7-b973-4cad-b6eb-3064d4413b7c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:56.240224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e8bb7-b973-4cad-b6eb-3064d4413b7c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:56.242122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 570/1124] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:56.247413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:56.247814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:56.247969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:56.265055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:56.266252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:56.329767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 555/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:26:56.353405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e8bb7-b973-4cad-b6eb-3064d4413b7c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:56.451224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e8bb7-b973-4cad-b6eb-3064d4413b7c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:56.470000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:56.485155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:56.486000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:56.486283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:56.486560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] PATH_INFO: `/v3/roles/e5de296de6a641e89949ba5a46f31cb1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:56.488039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] RBAC: Authorizing `identity:get_role(role_id=e5de296de6a641e89949ba5a46f31cb1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:56.495037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:56.495037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 571/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:56 2022] GET /identity/v3/roles/e5de296de6a641e89949ba5a46f31cb1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:56.496719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:56.552947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:56.596062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dc2f8-64ee-4063-a4b5-081ea06fb5c4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:56.629206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:56.630068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:56.630351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:56.630627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] PATH_INFO: `/v3/users/dff7f668779e4de38ab4ac828e5df2d5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:56.631830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] RBAC: Authorizing `identity:get_user(user_id=dff7f668779e4de38ab4ac828e5df2d5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:56.633775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:56.635427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 556/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:26:56 2022] GET /identity/v3/users/dff7f668779e4de38ab4ac828e5df2d5 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:56.641071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:56.737706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663f5f-f031-4e0e-8ec8-d88fbd1400b1 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:56.770392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:56.771057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:56.771344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:56.771620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:56.824131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:56.835640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:56.837066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:56.929872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 572/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:26:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:26:56.935964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:57.031023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78338f-5c5b-4092-9778-eda974e5fa93 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:57.053451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:57.064260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:57.065095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:57.065380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:57.065658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] PATH_INFO: `/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:57.066810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c69290eda4d94f11a4db3b7566a3ccc1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:57.068258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc8ad3-0d39-446e-98a3-9659496844e1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:57.069936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 557/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:26:56 2022] GET /identity/v3/projects/c69290eda4d94f11a4db3b7566a3ccc1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:57.075852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:57.158463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:57.170394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d086c-d9c6-4da6-a420-4d575a680562 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:57.205345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:57.206001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:57.206288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:57.206568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:57.207433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] RBAC: Adding query filter params (role.id=e5de296de6a641e89949ba5a46f31cb1, scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1, user.id=dff7f668779e4de38ab4ac828e5df2d5) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:26:57.207750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:57.209164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af776e-5e4c-4ab8-a930-479f4e630160 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:57.214838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 573/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:26:57 2022] GET /identity/v3/role_assignments?user.id=dff7f668779e4de38ab4ac828e5df2d5&scope.project.id=c69290eda4d94f11a4db3b7566a3ccc1&role.id=e5de296de6a641e89949ba5a46f31cb1 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:59.325139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a01b4-296e-4432-bf75-c84cdd0954f8 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:59.325662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a01b4-296e-4432-bf75-c84cdd0954f8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:59.325748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a01b4-296e-4432-bf75-c84cdd0954f8 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:59.327095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 558/1131] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:26:59.332215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:59.332604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:59.332770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:59.350326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:26:59.351575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:26:59.415656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 574/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:26:59.439209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a01b4-296e-4432-bf75-c84cdd0954f8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:59.537272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a01b4-296e-4432-bf75-c84cdd0954f8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:59.570022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:59.570797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:59.571113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:59.571394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:59.572565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:59.573993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f528f4-6838-419a-bd26-f97902f8f927 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:59.575602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 559/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:26:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:26:59.581292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:26:59.658572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:26:59.675315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2c019-4ec0-4cd6-b135-7ed9c4f2aba8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:26:59.713424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:26:59.714018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:26:59.714306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:26:59.714585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:26:59.715318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:26:59.716738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bbe3d-e68b-4a15-b265-ef4cb82f7d12 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:26:59.807256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 575/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:26:59 2022] POST /identity/v3/projects => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:27:01.924624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6444d-7a8e-48b5-bb12-98ba27df673c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:01.925187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6444d-7a8e-48b5-bb12-98ba27df673c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:01.925187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6444d-7a8e-48b5-bb12-98ba27df673c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:01.926529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 560/1135] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:27:01.932239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:01.932661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:01.932810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:01.950039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:01.951259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:02.015735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 576/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:02.040540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6444d-7a8e-48b5-bb12-98ba27df673c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:02.098825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:02.140023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6444d-7a8e-48b5-bb12-98ba27df673c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:02.173559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:02.174340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:02.174623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:02.174976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:02.176089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:02.177503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc34e0-5565-47e9-afe5-e2c59f80be7c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:02.179306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 561/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:02.185033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:02.244208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:02.280586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55632043-b374-4b57-bf23-5e168385c4e2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:02.313664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:02.314323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:02.314607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:02.314883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:02.315618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:02.317031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295580e-075e-4897-bf67-6159c701a420 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:02.393886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 577/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:27:02 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:27:04.535247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4c8bf-11f7-453e-a703-1c64c6b5db54 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:04.535779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4c8bf-11f7-453e-a703-1c64c6b5db54 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:04.535873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4c8bf-11f7-453e-a703-1c64c6b5db54 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:04.537135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 562/1139] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:04.542703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:04.543140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:04.543303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:04.560708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:04.561888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:04.627369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 578/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:27:04.649997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4c8bf-11f7-453e-a703-1c64c6b5db54 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:04.746781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4c8bf-11f7-453e-a703-1c64c6b5db54 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:04.771203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:04.778362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:04.779155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:04.782770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:04.782770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:04.789988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:04.797554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:04.798824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:04.799907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 563/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:04.805469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:04.852545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:04.900730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1164332-9f25-4280-a62a-da16b92d5272 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:04.934461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:04.935390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:04.935679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:04.935954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:04.936690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:04.936996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:04.938339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:04.946041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 579/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:04.951814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:05.049069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2f94e-9f8f-4cff-98ba-e74a6537b70d None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:05.081343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:05.082195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:05.082483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:05.082767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] PATH_INFO: `/v3/users/b22bbdf01f994b16a02777a82fdc3dd7` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:05.095962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] RBAC: Authorizing `identity:get_user(user_id=b22bbdf01f994b16a02777a82fdc3dd7)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:05.097402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:05.099179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 564/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:04 2022] GET /identity/v3/users/b22bbdf01f994b16a02777a82fdc3dd7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:05.104977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:05.201810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626550b7-6b7e-460f-ab25-ec3d0d7209b3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:05.214542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:05.234903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:05.235582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:05.235871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:05.236154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:05.300593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:05.301973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:05.396331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 580/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:27:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 19:27:05.402164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:05.447825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:05.500046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc1f1-135f-4236-b5d8-506e9f9ea489 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:05.531968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:05.532808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:05.533095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:05.533377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] PATH_INFO: `/v3/projects/863a5673ac694c83933903b94e2fe5c2` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:05.534545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] RBAC: Authorizing `identity:get_project(project_id=863a5673ac694c83933903b94e2fe5c2)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:05.536015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa2c53-54c4-49d2-b5f0-218fdbd25705 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:05.538207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 565/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:05 2022] GET /identity/v3/projects/863a5673ac694c83933903b94e2fe5c2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:05.544077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:05.558398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:05.640141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc3ed-5d92-4830-830b-48d53db185de None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:05.672633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:05.673276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:05.673564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:05.673840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:05.674667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=863a5673ac694c83933903b94e2fe5c2, user.id=b22bbdf01f994b16a02777a82fdc3dd7) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:05.675010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:05.676430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e489dac-86f6-432d-81a9-0e03b519c1b1 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:05.682308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 581/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:27:05 2022] GET /identity/v3/role_assignments?user.id=b22bbdf01f994b16a02777a82fdc3dd7&scope.project.id=863a5673ac694c83933903b94e2fe5c2&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:07.764365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f482-918a-4e04-a2ca-a29061f8e0fe None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:07.764899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f482-918a-4e04-a2ca-a29061f8e0fe None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:07.764899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f482-918a-4e04-a2ca-a29061f8e0fe None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:07.766263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 566/1147] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:07.771342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:07.771814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:07.771923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:07.788455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:07.789614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:07.852928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 582/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:07.876940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0f482-918a-4e04-a2ca-a29061f8e0fe None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:07.976691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0f482-918a-4e04-a2ca-a29061f8e0fe None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:07.989832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:08.008500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:08.009275 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:08.009561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:08.009838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:08.019504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:08.027665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:08.028595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:08.029487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 567/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:08.035232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:08.043681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:08.130688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d9dd-379a-4fc9-bfc6-64c078af20ad None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:08.163918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:08.164582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:08.164871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:08.165152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:08.165906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:08.166219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:08.167614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:08.175743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 583/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:08.181411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:08.279335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22229c-723b-457b-9425-8b43382236ad None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:08.311461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:08.312315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:08.312598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:08.312873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] PATH_INFO: `/v3/users/b22bbdf01f994b16a02777a82fdc3dd7` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:08.314036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] RBAC: Authorizing `identity:get_user(user_id=b22bbdf01f994b16a02777a82fdc3dd7)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:08.315483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:08.317080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 568/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:08 2022] GET /identity/v3/users/b22bbdf01f994b16a02777a82fdc3dd7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:08.323493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:08.389149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:08.421917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab34cd-3aaf-4b06-82fc-f59f9e478ae3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:08.454520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:08.455403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:08.455690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:08.455972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] PATH_INFO: `/v3/projects/863a5673ac694c83933903b94e2fe5c2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:08.457139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] RBAC: Authorizing `identity:get_project(project_id=863a5673ac694c83933903b94e2fe5c2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:08.458565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0e5ec-6a8f-481d-9070-81cd21985ec5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:08.460321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 584/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:08 2022] GET /identity/v3/projects/863a5673ac694c83933903b94e2fe5c2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:08.466236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:08.508094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:08.561949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12460dc-7899-4045-a98c-324a193e6744 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:08.597794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:08.599029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:08.599322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:08.599636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] PATH_INFO: `/v3/projects/863a5673ac694c83933903b94e2fe5c2/users/b22bbdf01f994b16a02777a82fdc3dd7/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:08.602233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] RBAC: Authorizing `identity:create_grant(project_id=863a5673ac694c83933903b94e2fe5c2, user_id=b22bbdf01f994b16a02777a82fdc3dd7, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:08.603694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb500da-5e3d-4b30-9e95-cc5a3fdbb1dd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:08.662063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 569/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:27:08 2022] PUT /identity/v3/projects/863a5673ac694c83933903b94e2fe5c2/users/b22bbdf01f994b16a02777a82fdc3dd7/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:27:10.734556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403451-9b93-4bcb-960c-752ed076fca9 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:10.735115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403451-9b93-4bcb-960c-752ed076fca9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:10.735275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403451-9b93-4bcb-960c-752ed076fca9 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:10.736646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 585/1154] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:10.742299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:10.742776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:10.742865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:10.759132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:10.760309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:10.829689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 570/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:10.852975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403451-9b93-4bcb-960c-752ed076fca9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:10.951509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403451-9b93-4bcb-960c-752ed076fca9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:10.984473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:10.985296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:10.985533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:10.985807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:10.988384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:10.996414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:11.003894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:11.005044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:27:11.006254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 586/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:11.010824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:11.040875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:11.106914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7f169-d3d9-484c-86fb-aabc526f848a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:11.138768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:11.139448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:11.139733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:11.140008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:11.140750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:11.141059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:11.142413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:11.150650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 571/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:11.156255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:11.252307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d5ffa-6a19-4724-ad7b-20b444ee79df None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:11.285583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:11.286465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:11.286748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:11.287053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] PATH_INFO: `/v3/users/b22bbdf01f994b16a02777a82fdc3dd7` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:11.288777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] RBAC: Authorizing `identity:get_user(user_id=b22bbdf01f994b16a02777a82fdc3dd7)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:11.290184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:11.291805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 587/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:11 2022] GET /identity/v3/users/b22bbdf01f994b16a02777a82fdc3dd7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:11.297747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:11.387035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:11.393059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc9bd0-e209-4061-89a6-88f2c474a57c None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:11.425930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:11.426564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:11.426846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:11.427149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:11.491369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:11.492791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:11.579642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:11.587201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 572/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:27:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:27:11.592939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:11.653166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:11.689630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a34d-cc51-4fab-a983-292475e0d548 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:11.722333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:11.723199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:11.723479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:11.723751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] PATH_INFO: `/v3/projects/863a5673ac694c83933903b94e2fe5c2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:11.724899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] RBAC: Authorizing `identity:get_project(project_id=863a5673ac694c83933903b94e2fe5c2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:11.726299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd087-8b38-4433-bdbc-dbb784df80c4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:11.727983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 588/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:11 2022] GET /identity/v3/projects/863a5673ac694c83933903b94e2fe5c2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:11.733867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:11.827544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cba26-80d2-4a29-8073-465770f98fa4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:11.859427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:11.860001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:11.860282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:11.860556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:11.861377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=863a5673ac694c83933903b94e2fe5c2, user.id=b22bbdf01f994b16a02777a82fdc3dd7) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:11.861690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:11.863126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f12fd-b573-4386-b898-b91cd5fa97c6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:11.869190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 573/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:27:11 2022] GET /identity/v3/role_assignments?user.id=b22bbdf01f994b16a02777a82fdc3dd7&scope.project.id=863a5673ac694c83933903b94e2fe5c2&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:13.963009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0ce5-c30b-4072-99aa-1197a6ca6e05 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:13.963546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0ce5-c30b-4072-99aa-1197a6ca6e05 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:13.963546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e0ce5-c30b-4072-99aa-1197a6ca6e05 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:13.964920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 589/1162] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:13.969923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:13.970303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:13.970479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:13.987297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:13.988464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:14.038816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:14.051742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 574/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:14.073488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e0ce5-c30b-4072-99aa-1197a6ca6e05 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:14.171153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e0ce5-c30b-4072-99aa-1197a6ca6e05 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:14.199316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:14.204043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:14.204684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:14.204971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:14.205250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:14.272164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:14.273615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:14.367575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 590/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:27:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:27:14.373474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:14.467111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da51f84-64f3-4366-b2b0-665a1b400006 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:14.498496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:14.499305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:14.499589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:14.499862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:14.506750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:14.508194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-040f70ef-ca2c-4fd1-b233-06fc201b71c9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:14.522531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:14.523795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:14.524687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 575/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:27:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:14.529443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:14.558389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:14.625687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113b37-4da2-46d7-952a-59dc08992ecf None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:14.659011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:14.659675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:14.659957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:14.660233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:14.660950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:14.661260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:14.662650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c25b2c-fef4-43ab-84fc-c7cfdc7c6ee2 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:14.669600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 591/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Thu Jun 16 19:27:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:16.733084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f089a-7038-4523-a0a8-9a63863bc1e9 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:16.733649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f089a-7038-4523-a0a8-9a63863bc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:16.733649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f089a-7038-4523-a0a8-9a63863bc1e9 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:16.735056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 576/1167] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:16.740010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce736ab-6835-495b-9cc8-0fe30c664871 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:16.740431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce736ab-6835-495b-9cc8-0fe30c664871 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:16.740615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce736ab-6835-495b-9cc8-0fe30c664871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:16.757348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3ce736ab-6835-495b-9cc8-0fe30c664871 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:16.758516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3ce736ab-6835-495b-9cc8-0fe30c664871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:16.822291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 592/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:16.845225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f089a-7038-4523-a0a8-9a63863bc1e9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:16.893719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:16.944444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f089a-7038-4523-a0a8-9a63863bc1e9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:16.975880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:16.976525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:16.976813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:16.977094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:16.977829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:16.979215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37e08a-b72b-4d31-89c9-442fc871a22a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:17.051220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 577/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Thu Jun 16 19:27:16 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:27:19.170539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afc36-618c-4fae-8a8d-a18f6eb0c729 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:19.170944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afc36-618c-4fae-8a8d-a18f6eb0c729 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:19.171134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afc36-618c-4fae-8a8d-a18f6eb0c729 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:19.172520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 593/1170] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:19.178122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:19.178535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:19.178693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:19.195470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:19.196652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:19.294483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 578/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:19.317502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438afc36-618c-4fae-8a8d-a18f6eb0c729 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:19.344414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:19.416305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438afc36-618c-4fae-8a8d-a18f6eb0c729 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:19.448141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:19.448985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:19.449271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:19.449552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] PATH_INFO: `/v3/domains/6cedcdb438d1486cbf1c0f8f6af554cf` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:19.460010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6cedcdb438d1486cbf1c0f8f6af554cf)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:19.461438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf1c3d-d9f4-4432-bb32-01cba42b6856 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:19.463071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 594/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Thu Jun 16 19:27:19 2022] GET /identity/v3/domains/6cedcdb438d1486cbf1c0f8f6af554cf => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:19.468745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:19.507262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:19.564415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf902-15a7-46ca-9396-151bc88201e9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:19.598618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:19.599446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:19.599728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:19.600005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:19.600733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:19.602164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a9538-9df5-442d-aba4-b56cf75f9a5e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:19.691732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 579/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:27:19 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:27:21.818912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404414-d016-4e83-a9c7-6041f0f7d376 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:21.819530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404414-d016-4e83-a9c7-6041f0f7d376 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:21.819530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404414-d016-4e83-a9c7-6041f0f7d376 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:21.820855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 595/1174] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:21.826024 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:21.826449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:21.826584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:21.842815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:21.844032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:21.907291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 580/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:21.932001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59404414-d016-4e83-a9c7-6041f0f7d376 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:22.016466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:22.031011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59404414-d016-4e83-a9c7-6041f0f7d376 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:22.064112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:22.064959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:22.065242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:22.065517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] PATH_INFO: `/v3/domains/6cedcdb438d1486cbf1c0f8f6af554cf` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:22.066662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6cedcdb438d1486cbf1c0f8f6af554cf)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:22.068221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9046d09-6d8f-427e-8665-7085cbe1ab92 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:22.069859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 596/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Thu Jun 16 19:27:21 2022] GET /identity/v3/domains/6cedcdb438d1486cbf1c0f8f6af554cf => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:22.075646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:22.095832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:22.170736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aae64-cb92-413c-9606-5ca573d4d5d9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:22.203828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:22.204482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:22.204760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:22.205035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:22.205750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:22.207188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfad16b-a7fe-432c-b72e-15f3e47bdf38 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:22.283780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 581/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:27:22 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:27:24.455246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971eb9b-e127-43f8-a11a-334e2c295bb5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:24.455769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971eb9b-e127-43f8-a11a-334e2c295bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:24.455769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971eb9b-e127-43f8-a11a-334e2c295bb5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:24.457075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 597/1178] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:24.462837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:24.463256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:24.463440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:24.480361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:24.481526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:24.544681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 582/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:24.567752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971eb9b-e127-43f8-a11a-334e2c295bb5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:24.666292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971eb9b-e127-43f8-a11a-334e2c295bb5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:24.704511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:24.705355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:24.705591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:24.705871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:24.717300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:24.725574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:24.726786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:27:24.727908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 598/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:24.732821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:24.740847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:24.830067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd17642-a692-4a3d-843c-ff75633b1268 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:24.862016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:24.862660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:24.862967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:24.863246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:24.863985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:24.864329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:24.865685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:24.873325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 583/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:24.879083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:24.895183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:24.980750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747833b1-0ffc-4ff0-aca6-37455fceaf94 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:25.012959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:25.013847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:25.014132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:25.014409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] PATH_INFO: `/v3/users/3647338ab3254572a443b5ecdb350164` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:25.027005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] RBAC: Authorizing `identity:get_user(user_id=3647338ab3254572a443b5ecdb350164)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:25.028446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:25.030076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 599/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:24 2022] GET /identity/v3/users/3647338ab3254572a443b5ecdb350164 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:25.035976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:25.101900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:25.133739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524ec4c-8abe-4a7b-8b03-b86d40bb6c82 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:25.166405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:25.167083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:25.167371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:25.167651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:25.232319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:25.233693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:25.299407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:25.332074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 584/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:27:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:27:25.338048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:25.404995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:25.432774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603461f-3356-431b-83fc-eea2fb5332df None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:25.465795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:25.466637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:25.466945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:25.467229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] PATH_INFO: `/v3/projects/2ff68161e9814cc692ed6dde243031a9` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:25.468383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] RBAC: Authorizing `identity:get_project(project_id=2ff68161e9814cc692ed6dde243031a9)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:25.469800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3de1a7-926d-49b5-8399-2286da4186ab None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:25.471506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 600/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:25 2022] GET /identity/v3/projects/2ff68161e9814cc692ed6dde243031a9 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:25.477491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:25.571010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d3ddc-856f-44bf-a0ad-c9376f76c4ef None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:25.602863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:25.603520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:25.603804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:25.604078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:25.604901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=2ff68161e9814cc692ed6dde243031a9, user.id=3647338ab3254572a443b5ecdb350164) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:25.605212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:25.606615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a60aa-5008-4631-b2d8-325d46c2dfaa None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:25.611822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 585/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:27:25 2022] GET /identity/v3/role_assignments?user.id=3647338ab3254572a443b5ecdb350164&scope.project.id=2ff68161e9814cc692ed6dde243031a9&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:27.696548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b50cf-4eef-4bca-8b19-2b26c6c9f32c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:27.697081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b50cf-4eef-4bca-8b19-2b26c6c9f32c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:27.697081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b50cf-4eef-4bca-8b19-2b26c6c9f32c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:27.698448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 601/1186] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:27.703819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62291-627e-4566-9436-b5de78778285 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:27.704255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62291-627e-4566-9436-b5de78778285 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:27.704378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62291-627e-4566-9436-b5de78778285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:27.721059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cbf62291-627e-4566-9436-b5de78778285 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:27.722227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cbf62291-627e-4566-9436-b5de78778285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:27.747093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:27.785413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 586/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:27.808005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b50cf-4eef-4bca-8b19-2b26c6c9f32c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:27.908188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b50cf-4eef-4bca-8b19-2b26c6c9f32c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:27.940032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:27.940805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:27.941086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:27.941359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:27.949243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:27.952065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:27.960273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:27.961411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:27:27.962291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 602/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:27.967594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf62291-627e-4566-9436-b5de78778285 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:28.062030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf62291-627e-4566-9436-b5de78778285 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:28.095096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:28.095749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:28.096043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:28.096324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:28.097073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:28.097384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:28.098774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:28.106559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 587/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:28.112137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:28.209205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9659e-4e5c-4eb8-b25b-1a0a3388ad4a None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:28.240766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:28.241616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:28.241897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:28.242170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] PATH_INFO: `/v3/users/3647338ab3254572a443b5ecdb350164` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:28.243359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] RBAC: Authorizing `identity:get_user(user_id=3647338ab3254572a443b5ecdb350164)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:28.244775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:28.246370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 603/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:28 2022] GET /identity/v3/users/3647338ab3254572a443b5ecdb350164 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:28.252011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:28.277261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:28.353319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2d055-c0ec-4896-b4ca-9756c18ca5cf None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:28.386383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:28.387288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:28.387573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:28.387851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] PATH_INFO: `/v3/projects/2ff68161e9814cc692ed6dde243031a9` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:28.389016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] RBAC: Authorizing `identity:get_project(project_id=2ff68161e9814cc692ed6dde243031a9)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:28.390437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cf688-06b5-4326-b2f4-ae204f89eae0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:28.392153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 588/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:28 2022] GET /identity/v3/projects/2ff68161e9814cc692ed6dde243031a9 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:28.398130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:28.463074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:28.495158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50fec8-2a2d-40a3-a4da-852410a49625 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:28.527794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:28.528668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:28.528959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:28.529240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] PATH_INFO: `/v3/projects/2ff68161e9814cc692ed6dde243031a9/users/3647338ab3254572a443b5ecdb350164/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:28.531211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] RBAC: Authorizing `identity:create_grant(project_id=2ff68161e9814cc692ed6dde243031a9, user_id=3647338ab3254572a443b5ecdb350164, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:28.532642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0f25d-fd02-4603-881b-40c298c06b9d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:28.603532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 604/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:27:28 2022] PUT /identity/v3/projects/2ff68161e9814cc692ed6dde243031a9/users/3647338ab3254572a443b5ecdb350164/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:27:30.703046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d41d0-b8cf-4a6d-b4c7-bad561d4e5db None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:30.703602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d41d0-b8cf-4a6d-b4c7-bad561d4e5db None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:30.703602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d41d0-b8cf-4a6d-b4c7-bad561d4e5db None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:30.704947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 589/1193] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:27:30.710101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:30.710486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:30.710666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:30.726957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:27:30.728133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:27:30.795990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 605/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:27:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:27:30.819696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d41d0-b8cf-4a6d-b4c7-bad561d4e5db None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:30.906738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:30.916251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d41d0-b8cf-4a6d-b4c7-bad561d4e5db None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:30.948084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:30.948850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:30.949132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:30.949409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:30.959667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:27:30.967152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:27:30.968259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:27:30.969494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 590/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:27:30.975591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:31.031936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:31.072206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4435cc4-3f85-412c-82b4-f1fa9fcde289 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:31.104929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:31.105587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:31.105878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:31.106158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:31.106901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:31.107243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:31.108592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:31.116950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 606/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:27:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:31.122654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:31.220426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fce7f-ab75-47dc-9d12-5dc75a9f2778 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:31.252367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:31.253233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:31.253517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:31.253794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] PATH_INFO: `/v3/users/3647338ab3254572a443b5ecdb350164` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:31.254982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] RBAC: Authorizing `identity:get_user(user_id=3647338ab3254572a443b5ecdb350164)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:31.256408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:31.258002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 591/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Thu Jun 16 19:27:31 2022] GET /identity/v3/users/3647338ab3254572a443b5ecdb350164 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:31.263652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:31.364177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e103-4c0e-4387-bab0-2bdb0de80416 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:31.383374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:31.397318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:31.397968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:31.398255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:31.398533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:31.464235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:31.465701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:31.560576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 607/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:27:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:27:31.566400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:31.588943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:31.665718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2fa0-08ca-4276-9850-9115ac8ecd8e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:31.698404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:31.699277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:31.699565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:31.699844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] PATH_INFO: `/v3/projects/2ff68161e9814cc692ed6dde243031a9` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:31.701013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] RBAC: Authorizing `identity:get_project(project_id=2ff68161e9814cc692ed6dde243031a9)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:31.702456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196e56b-e3b6-4240-b1c9-3a7b3991b91e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:31.704174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 592/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Thu Jun 16 19:27:31 2022] GET /identity/v3/projects/2ff68161e9814cc692ed6dde243031a9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:31.711213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:27:31.731699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:31.805811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36376e95-c12a-4330-9049-0665dc9d2099 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:27:31.839014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:27:31.839607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:27:31.839892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:27:31.840172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:27:31.840998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=2ff68161e9814cc692ed6dde243031a9, user.id=3647338ab3254572a443b5ecdb350164) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:27:31.841317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:27:31.842727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba55efa-6e4e-4bf9-b63c-a2620511f581 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:27:31.848585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 608/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:27:31 2022] GET /identity/v3/role_assignments?user.id=3647338ab3254572a443b5ecdb350164&scope.project.id=2ff68161e9814cc692ed6dde243031a9&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:27:41.694271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=56 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:27:41.845947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=58 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:45.002589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f586fb-8ee5-4222-9e73-eb46586bb425 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:45.003203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f586fb-8ee5-4222-9e73-eb46586bb425 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:45.003203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f586fb-8ee5-4222-9e73-eb46586bb425 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:45.004663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 593/1201] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:28:45.009854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:45.010286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:45.010442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:45.027782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:28:45.029049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:28:45.093898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 609/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:28:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:28:45.117631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f586fb-8ee5-4222-9e73-eb46586bb425 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:45.217961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f586fb-8ee5-4222-9e73-eb46586bb425 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:45.250759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:45.251566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:45.251851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:45.252128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:45.262509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:45.263949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:45.280785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:45.282010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:45.288241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:45.288241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:45.288241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:45.288241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:45.288241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 594/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:45.288635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:45.391623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301883e-e65f-45e2-832e-6563afe06b63 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:45.419600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:45.425233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:45.426669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:45.426669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:45.426669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:45.427206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:45.427528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:45.428861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b28c-ca84-4212-8ad7-cf654c7d29b5 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:45.440553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 610/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:45.446520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:45.498839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:45.544676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3acbc6-7a95-4fa6-bbe0-c7dcd70c4a01 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:45.576959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:45.577624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:45.577912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:45.578194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:45.578913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:45.583115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df54e8-24f4-4f6d-81f9-3367fc168353 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:45.665589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 595/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Thu Jun 16 19:28:45 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:28:47.790030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368efa-637f-498c-a666-d75ec2cef670 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:47.790585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368efa-637f-498c-a666-d75ec2cef670 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:47.790585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368efa-637f-498c-a666-d75ec2cef670 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:47.792541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 611/1206] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:28:47.797597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:47.798011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:47.798165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:47.815745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:28:47.816937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:28:47.881528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 596/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:28:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:28:47.905050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368efa-637f-498c-a666-d75ec2cef670 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.006745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368efa-637f-498c-a666-d75ec2cef670 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.039484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.040323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.040554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.040953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.052061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:28:48.060115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:48.061267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:48.062347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 612/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:28:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:48.067392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.105779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:48.161107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0925-bb76-4b5b-975f-fc72cce06a5c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.195582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.196241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.196524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.196797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.198103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:48.198406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:48.199818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:48.207410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 597/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:48.213236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.263851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:48.312854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9774a8-bbe1-41d2-bcb3-6b74eae0abe3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.345485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.349402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.349402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.349402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.357454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:48.358997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:48.377246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:48.378548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:48.379520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:48.379520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:48.379520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:48.379520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:48.379520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 613/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:48.384389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.479735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:48.482621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a54a72-d168-4fd2-9bcc-cfbb7b604ed1 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.515183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.515912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.516212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.516481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.517202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:48.517512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:48.518879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:48.529947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 598/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:48.535763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.634080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87026ce-1ab1-4b72-a1f8-4ec7bbcd7b05 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.635624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:48.672747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.673548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.673830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.674104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.681153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:48.682891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:28:48.697421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:48.698571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:48.699724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 614/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:48.704364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.800742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f262-4b3e-490d-bcb5-446b4d0647e8 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.832908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.833564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.833846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.834123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:48.834975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:48.835287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:48.836755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:48.853490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 599/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:48.859696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:48.960130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64870ba-dcaf-4da4-a7c0-019cf67d1c7f None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:48.992245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:48.992886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:48.993168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:48.993447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.022486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.058461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.060201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.154109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 615/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:28:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:28:49.160108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:49.178544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.260824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811abb9-9d3b-4f7f-8d90-5471b7bc777b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:49.294019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:49.294791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:49.295107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:49.295403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.305294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.306746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:49.324694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:49.325759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:49.326767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:49.326767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:49.326767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:49.326767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:49.326767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 600/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:49.331812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:49.396643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.434297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629b408-6ed3-4bce-a49a-18f5354d0671 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:49.468291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:49.468957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:49.469253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:49.469519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.470235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:49.470546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.471964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.483115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 616/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:28:49.488389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:49.555541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.586441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16e4d-9087-4933-bb52-462d8723793e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:49.618119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:49.619020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:49.619834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:49.620108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.626466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.627916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:28:49.642205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:49.643311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:49.644396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 601/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:49.649190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:49.748136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f6c3-8861-4233-bc70-dbd02727e70b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:49.769773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.781010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:49.781665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:49.781993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:49.782271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.783094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:49.783409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.784855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed7187-76c8-4bda-a765-988d7583754a None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.797079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 617/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:49.803345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:49.900262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab3ba5-fdfc-4c1b-bb94-6270ec429566 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:49.929923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:49.936370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:49.937019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:49.937308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:49.937590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:49.938425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=02a46335d4b248048b82d04e3665fe55) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:49.938742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:49.940197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751155b-71d4-42e8-982c-a89582a3fc73 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:49.945468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 602/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:28:49 2022] GET /identity/v3/role_assignments?user.id=02a46335d4b248048b82d04e3665fe55&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:52.037672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d63e-fda8-485a-ba51-00e1f8484089 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.038239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d63e-fda8-485a-ba51-00e1f8484089 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.038239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d63e-fda8-485a-ba51-00e1f8484089 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.039623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 618/1220] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:28:52.044884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.045291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.045450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.062287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:28:52.067510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:28:52.129406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 603/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:28:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:28:52.151856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1d63e-fda8-485a-ba51-00e1f8484089 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.251117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1d63e-fda8-485a-ba51-00e1f8484089 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:52.283172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:52.283957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.284241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.284518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.296384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:28:52.305203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:52.306352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:52.307628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 619/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:52.313485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.408441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466c579-442f-404c-82bf-3caa8d6e3848 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:52.431492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:52.441953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:52.442610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.442895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.443213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.443960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] RBAC: Adding query filter params (name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:52.444267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:52.445623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:52.453239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 604/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:52.459053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.502475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:52.557639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4babc2d-0dfa-48e5-83d2-737640f89ae8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:52.589865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:52.590644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.590953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.591240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.601323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:52.602761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:52.619750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:52.620849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:52.622168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:52.622168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:52.622168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:52.622168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:52.622168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 620/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:52.626671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.724216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937775c-bd3c-4217-8c94-dafe70f61819 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:52.757046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:52.757685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.757972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.758254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.758997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:52.759310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:52.761222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:52.772537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 605/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:52.778192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.868878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:52.875829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ce55-e476-4c3d-b7f2-77b9609bd785 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:52.907311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:52.908110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:52.908393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:52.908671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:52.915959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:52.917914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:28:52.932593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:52.933720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:52.935145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 621/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:52.939909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:52.986460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:53.039332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4788-b3ef-4148-9fe8-3407c715f0cd None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.072103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.072774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.073061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.073411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.074216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:53.074544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.075994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:53.093790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 606/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:28:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:53.099324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:53.200614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d41d9-443a-4e9d-8955-cb1b8334e720 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.233850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.234639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.234942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.235229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.246215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.247701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:53.264067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:53.265283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:53.266500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:53.266500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:53.266500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:53.266500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:53.266500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 622/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:53.271152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:53.370208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9880e-1020-4945-8cf6-05e74905b0ca None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.372549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:53.406936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.407681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.407967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.408249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.408967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:53.409271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.410625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:53.422386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 607/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:53.427972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:53.437425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:53.526626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a039270-a36a-49f7-94a4-910973ad3442 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.558830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.559955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.560217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.560492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.566957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.568402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:28:53.582647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:53.583758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:53.584921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 623/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:53.590233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:53.688663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c8e8-f6fa-440e-ab44-dbe8f8113e83 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.719928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.720571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.720855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.721132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.721989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:53.722301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.723766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa684f-738b-4c6b-a6f9-03916cf429e1 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:53.735760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 608/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:53.741677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:53.805191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:53.838415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c801-b3fb-47ca-94dc-02a7a82c9011 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:53.872272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:53.873209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:53.873491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:53.873769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/02a46335d4b248048b82d04e3665fe55/roles/0c3d0a5c22704753b6cfefbb565ab0c5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:53.887520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=02a46335d4b248048b82d04e3665fe55, role_id=0c3d0a5c22704753b6cfefbb565ab0c5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:53.889041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40fd86-704a-4d36-bff3-36ee43537e06 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:54.026533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 624/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:28:53 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/02a46335d4b248048b82d04e3665fe55/roles/0c3d0a5c22704753b6cfefbb565ab0c5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:28:56.140204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d18e24-2c83-4263-900f-00841bc340f4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.140774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d18e24-2c83-4263-900f-00841bc340f4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.140774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d18e24-2c83-4263-900f-00841bc340f4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.142107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 609/1233] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:28:56.147313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.147711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.147875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.164267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:28:56.165731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:28:56.231606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 625/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:28:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:28:56.255139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d18e24-2c83-4263-900f-00841bc340f4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:56.280391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:56.355043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d18e24-2c83-4263-900f-00841bc340f4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:56.396388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:56.397102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.397389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.397668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.408248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:28:56.415883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:56.417490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:56.418792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 610/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Thu Jun 16 19:28:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:56.423663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:56.468544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:56.519556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d14fb-8f57-49b5-805f-8c4acef2a0c3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:56.552207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:56.552880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.553170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.553454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.554209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:56.554523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:56.555922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:56.565047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 626/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Thu Jun 16 19:28:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:56.570736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:56.655455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:56.671142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae1330-998e-4260-97a1-b826158f68e9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:56.710934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:56.711808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.712091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.712364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.723469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:56.724911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:56.741950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:56.743020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:56.748761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:56.748761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:56.748761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:56.748761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:56.748761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 611/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:56.749134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:56.847356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff65cbf-ec66-4b12-b63a-5611575e5b04 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:56.856948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:56.880904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:56.881564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:56.881848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:56.882127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:56.882841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:56.883181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:56.884542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:56.895449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 627/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:56.901662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:56.998041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed90f-f6d4-4f7e-88b2-874cf294765b None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.031111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:57.031915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:57.032204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:57.032478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:57.035509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:57.040334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:57.042429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:28:57.057153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:57.058817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:28:57.060284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 612/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:57.065006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:57.163019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252bad-5a69-41a9-822e-17a97bf7eb47 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.196840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:57.197516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:57.197799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:57.198635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:57.199504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:57.204931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:57.204931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:57.217675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 628/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:28:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:57.223348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:57.321675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56576-e2b0-4a71-9af0-9b0498db41a2 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.355134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:57.355771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:57.356058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:57.356369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:57.415211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:57.425778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:57.428580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:57.530279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 613/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:28:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:28:57.536372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:57.564336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:57.634720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891820-1bdb-4cd2-a7d4-d2524180c1a6 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.667905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:57.668692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:57.668979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:57.669260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:57.678976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:57.680425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:28:57.697717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:57.698775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:57.699983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:28:57.699983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:28:57.699983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:28:57.699983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:57.699983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 629/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:57.704703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:57.806592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:57.812571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96cf3-c745-48a9-b3d1-c5ed7d4ddaab None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.845107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:57.845761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:57.846047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:57.846324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:57.847067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:57.847375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:57.848873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:57.860167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 614/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:57.873530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:57.943865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:28:57.964362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ba593-fb74-45e3-8955-44723c0c5359 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:57.996474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:58.000275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:58.000275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:58.000275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:58.004714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:58.006142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:28:58.020510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:28:58.021681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:28:58.022686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 630/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:28:58.028648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:58.126111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4a72-a8e8-4848-93c3-885847dad328 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:58.159381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:58.160036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:58.160320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:58.160635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:58.161433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:58.161748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:58.163228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6606e-0fdb-4f5a-a78a-a33b0c42f988 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:58.175206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 615/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:28:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:28:58.181204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:28:58.280004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2141e-7b16-4fa6-b859-45c90f11289c None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:28:58.312746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:28:58.313395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:28:58.313678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:28:58.313955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:28:58.314905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] RBAC: Adding query filter params (role.id=0c3d0a5c22704753b6cfefbb565ab0c5, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=02a46335d4b248048b82d04e3665fe55) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:28:58.315131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:28:58.316552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1921305-25e3-4b8d-b676-b5e3f8033be7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:28:58.322307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 631/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:28:58 2022] GET /identity/v3/role_assignments?user.id=02a46335d4b248048b82d04e3665fe55&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=0c3d0a5c22704753b6cfefbb565ab0c5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:00.486770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25382c-3d5d-4a6c-92f5-456957296874 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:00.492796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25382c-3d5d-4a6c-92f5-456957296874 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:00.492796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25382c-3d5d-4a6c-92f5-456957296874 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:00.492796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 616/1247] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:00.493741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:00.494120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:00.494300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:00.499425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:00.511764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:00.512924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:00.576734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 632/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:00.600133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25382c-3d5d-4a6c-92f5-456957296874 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:00.618753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:00.704167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25382c-3d5d-4a6c-92f5-456957296874 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:00.737251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:00.737969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:00.738255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:00.738534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:00.748469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:29:00.756200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:00.757837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:00.758692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 617/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:00.763544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:00.859975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabff83-19e4-4d10-b446-daa47083080b None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:00.887816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:00.892585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:00.893247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:00.893534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:00.893816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:00.894562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:00.894871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:00.896261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:00.904561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 633/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:00.910501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:00.987111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:01.008602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350215a-03df-4d8b-bef5-c763014cc0e3 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.040600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.041370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.041650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.041980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.052076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.053535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:01.070532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:01.071654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:01.072765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:01.072765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:01.072765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:01.072765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:01.072765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 618/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:01.078198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:01.175182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ed440-19fa-4d9e-bfa3-9fba17c279f3 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.208092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.208748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.209032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.209307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.210021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:01.210327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.211709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:01.223983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 634/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:01.229343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:01.326695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad1ae-86d3-440e-82b9-6cfe31030a87 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.352018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:01.359487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.360278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.360559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.360834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.367995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.373640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:29:01.383743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:01.384886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:01.387038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 619/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:01.391242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:01.446461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:01.491145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8fdf-154c-4c29-a709-faa3342ff12d None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.524926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.525603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.525884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.526166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.527034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:01.527351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.528760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:01.545153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 635/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:29:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:01.551073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:01.647763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1f6d-54ad-4036-bc54-0410359cad57 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.679464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.680104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.680382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.680655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.738296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:01.750609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.752090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:01.848513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 620/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:29:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:29:01.854826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:01.949527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6fd5-73e7-4042-96b1-7c865d8280c4 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:01.962801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:01.983485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:01.984276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:01.984589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:01.984873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:01.994857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:01.999073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:02.013814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:02.014861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:02.015919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:02.015919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:02.015919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:02.015919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:02.015919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 636/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:02.020855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:02.106611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:02.118462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eb3e3-7605-422b-a266-10c76ed57eb4 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:02.152032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:02.152697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:02.152983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:02.153261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:02.153995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:02.154287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:02.155680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:02.166928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 621/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:02.172942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:02.271114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0da6e6-918c-414a-ba73-fd5593f5bdf7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:02.305417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:02.306236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:02.306518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:02.306793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:02.313639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:02.315109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:29:02.328703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:02.329781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:02.331095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 637/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:02.335864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:02.435518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e7418-4f5d-415a-91ff-259964db30b0 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:02.468662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:02.469312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:02.469596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:02.469875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:02.470669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:02.471007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:02.472447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58425876-42f1-4fa6-a1e8-898eff2d751d None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:02.484552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 622/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:02.490572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:02.516161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:02.589795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85000e6b-ce2d-4fd2-8072-15a807c1811b None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:02.622008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:02.622658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:02.622967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:02.623248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:02.624082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=02a46335d4b248048b82d04e3665fe55) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:02.624401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:02.625861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3be2d-7e90-43d6-abd0-9aa2056150cf None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:02.631373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 638/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:29:02 2022] GET /identity/v3/role_assignments?user.id=02a46335d4b248048b82d04e3665fe55&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:04.718732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30062195-065f-4536-bc8f-8aa909c462ff None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:04.719295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30062195-065f-4536-bc8f-8aa909c462ff None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:04.719295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30062195-065f-4536-bc8f-8aa909c462ff None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:04.720663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 623/1261] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:04.725683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:04.726106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:04.726253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:04.743674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:04.744960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:04.810440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 639/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:04.835348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30062195-065f-4536-bc8f-8aa909c462ff None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:04.846716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:04.935136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30062195-065f-4536-bc8f-8aa909c462ff None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:04.967331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:04.968137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:04.968415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:04.968690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:04.978499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:29:04.986475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:04.988073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:04.989227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 624/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:04.993880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.012518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:05.091125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b3db1-2ba8-4c5b-944a-c7a8ea205f40 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.124607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.125269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.125556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.125837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.126653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:05.126989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.128359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.136204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 640/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:05.141833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.207854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:05.241083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d345c-7204-4dc9-9bdf-876a2d6ac1a9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.273714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.274500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.274788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.275102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.285837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.287823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:05.305721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:05.306780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:05.307960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:05.307960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:05.307960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:05.307960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:05.307960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 625/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:05.312979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.392621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:05.411437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efcd8-c41e-4702-949a-19820ac7f4ee None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.444597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.445316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.445601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.445883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.446636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:05.446971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.448361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.460568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 641/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:05.466678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.578080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb14db-4172-4759-a77c-883d87a2eec5 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.597062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:05.610824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.611698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.611983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.612266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.619667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.621174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:29:05.635487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:05.637128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:05.638361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 626/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:05.643046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.741600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159550-e719-4ff4-a5fe-bbddd734d2a7 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.774120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.774777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.775088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.775365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.776196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:05.777081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.778490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.786753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:05.794789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 642/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:05.800585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:05.899935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291725f0-2f62-4faf-8c7b-0c4f6b48152a None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:05.932189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:05.933601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:05.933888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:05.934164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:05.944597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:05.946043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:05.963198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:05.964243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:05.965322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:05.965322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:05.965322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:05.965322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:05.965322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 627/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:05.970102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:06.067281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee73e-6aa2-4009-a9e9-280997278b06 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:06.100562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:06.101223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:06.101514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:06.101794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:06.102511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:06.102821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:06.104222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:06.115419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 643/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:06.121516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:06.152818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:06.220842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771ebc3-14fb-41ca-9f2f-65f6bdbf1d35 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:06.255414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:06.256199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:06.256480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:06.256754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:06.263794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:06.265228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:29:06.279123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:06.280168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:06.281298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 628/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:06.286284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:06.344153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:06.383523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67f0fe-503f-4f39-9796-f4b64b2826cc None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:06.417125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:06.417721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:06.418003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:06.418276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:06.419095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:06.425835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:06.425835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bb1a2-8b6a-4b27-a68a-c2887b311187 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:06.437251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 644/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:06.443361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:06.532828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:06.541371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95037746-d68c-4432-b579-9be9b30fe7f9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:06.574513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:06.575424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:06.575706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:06.575982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] PATH_INFO: `/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/02a46335d4b248048b82d04e3665fe55/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:06.578059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] RBAC: Authorizing `identity:create_grant(project_id=a0871998ca834c5da02ec3173a0e7e37, user_id=02a46335d4b248048b82d04e3665fe55, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:06.579856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8e29e-ec8d-4949-88b9-2dd6e72dce2c None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:06.683315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 629/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Thu Jun 16 19:29:06 2022] PUT /identity/v3/projects/a0871998ca834c5da02ec3173a0e7e37/users/02a46335d4b248048b82d04e3665fe55/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:29:08.790175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da1d6-1561-4883-8cec-563111c0b783 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:08.790788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da1d6-1561-4883-8cec-563111c0b783 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:08.790788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da1d6-1561-4883-8cec-563111c0b783 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:08.792624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 645/1274] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:08.797728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:08.798116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:08.798296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:08.815553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:08.816738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:08.881370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 630/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:08.904940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0da1d6-1561-4883-8cec-563111c0b783 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.005621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0da1d6-1561-4883-8cec-563111c0b783 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.024006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:09.040445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.041178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.041483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.041757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.051436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 19:29:09.059735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:09.061100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:09.062370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 646/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Thu Jun 16 19:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:09.067151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.147590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:09.164692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bdaf1-ef8e-48e0-9a89-74c9f7450a60 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.197716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.199020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.199313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.199595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.200354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:09.200662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:09.202035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:09.209911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 631/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:09.215920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.315805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e985bee-dc34-42eb-adb7-41ee842063b0 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.347675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.348455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.348740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.349017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.359714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:09.361161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:09.379646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:09.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:09.384014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:09.384014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:09.384014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:09.384014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:09.384014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 647/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 19:29:09.386780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.484666 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c562b-05f7-4680-95e2-81a0a2fcc57e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.517433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.518089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.518375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.518656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.519411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:09.519721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:09.521083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:09.543981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 632/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:09.549585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.580767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:09.647743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ff42-f6f9-438b-aeb1-09b0dd4b72d8 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.681692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.682635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.682944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.683224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.690641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:09.692355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 19:29:09.706429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:09.707580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:09.708975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 648/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:09.713378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.718079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:09.811798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259ae78-a4c1-4575-8403-904f07cffbde None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:09.846056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:09.846663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:09.846965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:09.847246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:09.848086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:09.848399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:09.849832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:09.866905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 633/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:09.872609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:09.959166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:09.968744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cf49-f503-466d-bb68-999d416352aa None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.000739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.001548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.001899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.002505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.067340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.068766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.166970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:10.176995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 649/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Thu Jun 16 19:29:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 19:29:10.182914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:10.243290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:10.281296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08baca3-4e63-462d-86d3-47d79892247e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.313577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.314374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.314658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.314952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.325618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.327073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 19:29:10.344065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:10.345188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:10.346439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 19:29:10.346439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 19:29:10.346439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 19:29:10.346439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:10.346439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 634/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:10.350900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:10.446202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada31cb-57d7-48e7-ab41-3a9aef3c5246 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.480696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.481376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.481659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.481936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.482685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:10.483032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.484389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.497309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 650/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Thu Jun 16 19:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:10.504577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:10.602047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935e1d8-e96d-4da4-b5e7-1324e686ef2e None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.615000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:10.634150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.634947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.635239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.635516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.642400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.643870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 19:29:10.657702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:10.658688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: ERROR keystone.server.flask.application Jun 16 19:29:10.659723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 635/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Thu Jun 16 19:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:10.664677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:10.714534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:10.763321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9ea52-759e-474c-8bf6-067a86c11812 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.815345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.816011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.816300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.816580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.817372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:10.817686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.819144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c402f10-22a7-46bf-99cc-6495f5a835c4 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.832316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 651/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Thu Jun 16 19:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:10.838659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:10.936758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b25d1-60e6-47f2-af7e-da446bb13c95 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:10.968947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:10.969617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:10.969898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:10.970174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:10.971087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] RBAC: Adding query filter params (role.id=49095bd2b6a646058d983cc80858b8e3, scope.project.id=a0871998ca834c5da02ec3173a0e7e37, user.id=02a46335d4b248048b82d04e3665fe55) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:10.971400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:10.972853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b41e7-9a26-49df-9152-79faf1dd61b7 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:10.978539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 636/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Thu Jun 16 19:29:10 2022] GET /identity/v3/role_assignments?user.id=02a46335d4b248048b82d04e3665fe55&scope.project.id=a0871998ca834c5da02ec3173a0e7e37&role.id=49095bd2b6a646058d983cc80858b8e3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:13.163303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060acbc-78b4-4da7-9f2a-3610c7601595 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:13.163834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060acbc-78b4-4da7-9f2a-3610c7601595 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:13.163834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060acbc-78b4-4da7-9f2a-3610c7601595 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:13.165565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 652/1288] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:13.173215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:13.176016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:13.176016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:13.191678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:13.192895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:13.222171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:13.258043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 637/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:13.281666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060acbc-78b4-4da7-9f2a-3610c7601595 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:13.379998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060acbc-78b4-4da7-9f2a-3610c7601595 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:13.390644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:13.412440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:13.413233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:13.413587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:13.413789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:13.414430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:13.415810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:13.427300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:13.428901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:13.430031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 653/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:29:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:13.435011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:13.540179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07958d-fdee-481c-a664-ce00e2238e79 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:13.574784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:13.575477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:13.575762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:13.576037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:13.576747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:13.577054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:13.578432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b6cc0-b77b-44fc-b38a-9c7257af1e12 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:13.583190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 638/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Jun 16 19:29:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:13.589109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:13.686199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993df16-63b0-42b1-92e6-b81fd32acb74 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:13.719321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:13.719978 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:13.720266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:13.720545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] PATH_INFO: `/v3/services` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:13.721274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:13.721584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:13.722958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a606-c73b-409d-8a20-fc37e5e02e7d None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:13.727344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 654/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Jun 16 19:29:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:29:15.833313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981fe74-2f7f-4d99-9b76-a72463c37610 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:15.833857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981fe74-2f7f-4d99-9b76-a72463c37610 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:15.833857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981fe74-2f7f-4d99-9b76-a72463c37610 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:15.835252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 639/1293] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:15.840297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b2f27-5c2f-4dbd-9a11-dd3887acc298 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:15.842148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b2f27-5c2f-4dbd-9a11-dd3887acc298 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:15.842148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b2f27-5c2f-4dbd-9a11-dd3887acc298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:15.862031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ca9b2f27-5c2f-4dbd-9a11-dd3887acc298 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:15.863258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ca9b2f27-5c2f-4dbd-9a11-dd3887acc298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:15.892565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:15.927288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 655/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 19:29:15.951000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981fe74-2f7f-4d99-9b76-a72463c37610 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:15.981649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:16.048117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981fe74-2f7f-4d99-9b76-a72463c37610 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:16.080512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:16.081157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:16.081439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:16.081715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:16.082440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:16.083930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0cb86-c1c1-48ed-ba63-2a2b9d6e1ea6 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:16.166745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 640/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Thu Jun 16 19:29:15 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:29:18.280918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e42e-5174-4594-a6f9-48d862274f25 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.281449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e42e-5174-4594-a6f9-48d862274f25 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.281449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66e42e-5174-4594-a6f9-48d862274f25 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.282840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 656/1296] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:18.288117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.288546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.288695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.305133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:18.306299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:18.377669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 641/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:18.401571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66e42e-5174-4594-a6f9-48d862274f25 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:18.499691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66e42e-5174-4594-a6f9-48d862274f25 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:18.532788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:18.533595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.533924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.534198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.534846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:18.536301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:18.548211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:18.552950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:18.554349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 657/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:29:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:18.556188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:18.575566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:18.653568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd360ff-c3eb-4e68-b67a-e5d80ef820f3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:18.685877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:18.686525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.686810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.687122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.687825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:18.688131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:18.689483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:18.694387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 642/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Jun 16 19:29:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:29:18.701057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:18.731464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:18.798116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cab9b7-b480-4b03-b43f-bfabeb4c6ebc None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:18.832183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:18.832832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.833122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.833402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.834193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] RBAC: Adding query filter params (interface=public, service_id=17332a903a8642b7a695da88cca0e337, region_id=RegionOne) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:18.834510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:18.835903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e603ad1-54da-4ed5-ac98-18a21d5184c9 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:18.840884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 658/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Thu Jun 16 19:29:18 2022] GET /identity/v3/endpoints?service_id=17332a903a8642b7a695da88cca0e337&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:18.847158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:18.926059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:18.944062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efb9f7-1eb6-43fe-99ca-6eb76c4705b9 None admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:18.977952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:18.978609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:18.978898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:18.979212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:18.979897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:18.981825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e1fa7-9fbd-4ee6-aecd-47665a555186 None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:18.988449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 643/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Thu Jun 16 19:29:18 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:29:21.099753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bca5-e0de-4615-8bf9-9742d07d3e4c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:21.100318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bca5-e0de-4615-8bf9-9742d07d3e4c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:21.100318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bca5-e0de-4615-8bf9-9742d07d3e4c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:21.101756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 659/1302] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:29:21.106905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:21.107344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:21.107502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:21.124360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:29:21.125553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:29:21.191400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 644/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:29:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 19:29:21.220955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128bca5-e0de-4615-8bf9-9742d07d3e4c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:21.314201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:21.320907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128bca5-e0de-4615-8bf9-9742d07d3e4c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:21.353872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:21.354599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:21.354881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:21.355184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:21.355876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:21.357227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 19:29:21.368718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 19:29:21.369903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application with Lock( Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self._enter() Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application return self.creator() Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application created_value = creator( Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: ERROR keystone.server.flask.application Jun 16 19:29:21.371180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 660/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Thu Jun 16 19:29:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 19:29:21.378124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:21.453178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:21.475200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8127-56b1-4e31-93ab-d24e1ed5abe8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:21.506864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:21.507549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:21.507837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:21.508118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] PATH_INFO: `/v3/services` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:21.508830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] RBAC: Adding query filter params (name=placement) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:29:21.509140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:21.510521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7b7a1-77bb-4531-b2f8-1721a23159af None admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:21.516041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 645/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Thu Jun 16 19:29:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:29:21.521902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:29:21.623318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4237e2f-ba34-400c-8c62-e13757211032 None admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:29:21.655806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:29:21.656486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:29:21.656773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:29:21.657056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:29:21.657798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:29:21.659197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52678e6-543c-48a0-a945-e656fb11123f None admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:29:21.738091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 661/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Thu Jun 16 19:29:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:29:31.549755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=30 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:29:31.733693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:13.038947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821c5d-a08f-4d55-b5a6-8d6b81c50d2b None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:13.039525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821c5d-a08f-4d55-b5a6-8d6b81c50d2b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:13.039525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821c5d-a08f-4d55-b5a6-8d6b81c50d2b None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:13.046994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49922-c5a8-4251-8f73-8644750bbcaa None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:13.047529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49922-c5a8-4251-8f73-8644750bbcaa None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:13.047529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49922-c5a8-4251-8f73-8644750bbcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:13.051330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 646/1307] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:13.134948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4bf49922-c5a8-4251-8f73-8644750bbcaa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:13.136157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4bf49922-c5a8-4251-8f73-8644750bbcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:13.248337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 662/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:35:13.311511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:13.311795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:13.312056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:13.313427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 647/1309] 10.222.0.36 () {54 vars in 964 bytes} [Thu Jun 16 19:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:13.335682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45fd05-cbb2-40d0-b141-f796a655da7b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:13.335682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45fd05-cbb2-40d0-b141-f796a655da7b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:13.335682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45fd05-cbb2-40d0-b141-f796a655da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:13.488820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8d45fd05-cbb2-40d0-b141-f796a655da7b None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:13.490697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8d45fd05-cbb2-40d0-b141-f796a655da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:13.519675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:13.616421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 663/1310] 10.222.0.36 () {58 vars in 1061 bytes} [Thu Jun 16 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:13.617048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:13.618795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:13.743715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3604e78-ff6b-4daa-8612-7af1dd245c45 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:13.830020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:13.830641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:13.831040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:13.831491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:13.834496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:13.893404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:13.952295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:13.953737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06823a2-0233-4aaf-aca7-85987932ac1f service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:13.973435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 648/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:35:16.431489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:16.432004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:16.432004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:16.433350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 664/1312] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:16.438798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7e78-8cbc-4090-9fb9-f3a20394a115 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:16.439247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7e78-8cbc-4090-9fb9-f3a20394a115 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:16.439513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb7e78-8cbc-4090-9fb9-f3a20394a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:16.509365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d5fb7e78-8cbc-4090-9fb9-f3a20394a115 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:16.511158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d5fb7e78-8cbc-4090-9fb9-f3a20394a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:16.608019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 649/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:16.645529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:16.647118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:16.750265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33d02-8921-46e0-8559-fc0c0426a71e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:16.822582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:16.825245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:16.825897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:16.826174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:16.826448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:16.828131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:16.943802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:16.945259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6e649-6c5a-45d7-8be8-4a5a923ecbc6 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:16.968599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 665/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:19.521733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:19.522270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:19.522270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:19.523675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 650/1315] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:19.529485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0500-7fa8-4d06-93cf-c2d57bb1b553 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:19.529895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0500-7fa8-4d06-93cf-c2d57bb1b553 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:19.530043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0500-7fa8-4d06-93cf-c2d57bb1b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:19.593943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-934d0500-7fa8-4d06-93cf-c2d57bb1b553 None None] MFA Rules not processed for user `dffa90611fbe4b429e2703e120177f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:19.595155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-934d0500-7fa8-4d06-93cf-c2d57bb1b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:19.647892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:19.690746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 666/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:19 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:19.729772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:19.731364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:19.741072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:19.832413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b2476-5c66-48ed-9a8b-f6fc9c9232e3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:19.903020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:19.903665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:19.903947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:19.904223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:19.905895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:19.966676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:20.012091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:20.013513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64545e5e-4874-4649-b44f-eb0d71167eb1 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:20.031318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 651/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:22.889487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:22.890059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:22.890059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:22.891453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 667/1318] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:22.896652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a035-6744-4e9d-9c96-b6e3b5e660ce None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:22.897061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a035-6744-4e9d-9c96-b6e3b5e660ce None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:22.897233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a035-6744-4e9d-9c96-b6e3b5e660ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:22.960475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e4b8a035-6744-4e9d-9c96-b6e3b5e660ce None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:22.961683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e4b8a035-6744-4e9d-9c96-b6e3b5e660ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:23.055905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 652/1319] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:23.100553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:23.102120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:23.203685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661ac10-3ee0-4cd1-a80e-53b5c9d8a0a1 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:23.274701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:23.275369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:23.275654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:23.275930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:23.277570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:23.284790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:23.384496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:23.386158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe989c8-9468-4c7d-bd04-cd61039b1b9c service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:23.413687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 668/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:26.487821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:26.488570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:26.488570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:26.489789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 653/1321] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:26.495398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e3a7-8938-423e-b8c4-40756741f914 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:26.495848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e3a7-8938-423e-b8c4-40756741f914 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:26.495965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e3a7-8938-423e-b8c4-40756741f914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:26.562376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d667e3a7-8938-423e-b8c4-40756741f914 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:26.563638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d667e3a7-8938-423e-b8c4-40756741f914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:26.623580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:26.666713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 669/1322] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:26.702671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:26.704267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:26.724993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:26.818105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6a125-226f-4afe-a5e3-c3e70d26861c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:26.893690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:26.894330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:26.894611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:26.894884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:26.896541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:26.968403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:27.004971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:27.006399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd356a-a97b-4ade-b2cb-5a9258f3c1e6 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:27.026141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 654/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:29.212677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:29.213187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:29.213187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:29.214528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 670/1324] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:29.219962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3e88-6499-416b-b671-e68913bc9ffa None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:29.220421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3e88-6499-416b-b671-e68913bc9ffa None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:29.220520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3e88-6499-416b-b671-e68913bc9ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:29.283371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d6cf3e88-6499-416b-b671-e68913bc9ffa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:29.284574 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d6cf3e88-6499-416b-b671-e68913bc9ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:29.383936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 655/1325] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:35:29.420204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:29.421731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:29.523789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe45a1-c04d-4976-b280-e3c4019a1d00 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:29.601473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:29.602126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:29.602408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:29.602681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:29.605137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:29.621643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:29.723324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:29.724771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f3cb3-f62a-432a-a425-cfffa0da1ccf service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:29.743389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 671/1326] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:31.933197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:31.933706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:31.933825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:31.935441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 656/1327] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:31.940711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4e8f-90bd-422b-ae54-7579a0bc2b31 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:31.941100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4e8f-90bd-422b-ae54-7579a0bc2b31 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:31.941266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4e8f-90bd-422b-ae54-7579a0bc2b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:32.004037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-89ae4e8f-90bd-422b-ae54-7579a0bc2b31 None None] MFA Rules not processed for user `dffa90611fbe4b429e2703e120177f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:32.005232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-89ae4e8f-90bd-422b-ae54-7579a0bc2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:32.067527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:32.099521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 672/1328] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:32.137032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:32.138582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:32.170114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:32.244015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155257f-eb53-4ffe-b8f5-5ec6356b8db7 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:32.335530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:32.336284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:32.336639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:32.336988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:32.338770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:32.418597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:32.450510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:32.452982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a61eb2-767a-493d-aa28-5d8005d68da0 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:32.471345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 657/1329] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:35.116438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:35.117052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:35.117052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:35.118387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 673/1330] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:35.123613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc207f-f66e-4007-b992-670e65a80bb3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:35.124039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc207f-f66e-4007-b992-670e65a80bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:35.124174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc207f-f66e-4007-b992-670e65a80bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:35.190412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-00cc207f-f66e-4007-b992-670e65a80bb3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:35.191638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-00cc207f-f66e-4007-b992-670e65a80bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:35.286700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 658/1331] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:35.322992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:35.324530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:35.427776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63497b-db3e-4f1d-b5d8-619d11a25b4c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:35.500468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:35.501118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:35.501402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:35.501678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:35.503331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:35.529702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:35.610286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:35.611742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e652506-4c3b-4070-beef-6b61e00334cf service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:35.629165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 674/1332] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:35:37.783391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9581a2-fb74-4913-aed8-4143ce11337c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:37.783936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9581a2-fb74-4913-aed8-4143ce11337c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:37.783936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9581a2-fb74-4913-aed8-4143ce11337c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:37.785325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 659/1333] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:37.791249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc28a-678b-4055-8ad9-cd52974f3950 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:37.791693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc28a-678b-4055-8ad9-cd52974f3950 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:37.791825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcc28a-678b-4055-8ad9-cd52974f3950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:37.860991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-9abcc28a-678b-4055-8ad9-cd52974f3950 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:37.862205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9abcc28a-678b-4055-8ad9-cd52974f3950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:37.943338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:37.965068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 675/1334] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:38.020427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:38.020721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:38.024027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:38.024027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 660/1335] 10.222.0.36 () {54 vars in 964 bytes} [Thu Jun 16 19:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:38.028159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b52e-7a1f-4c3f-9cca-55a87f9a5164 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:38.028451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b52e-7a1f-4c3f-9cca-55a87f9a5164 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:38.028730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8b52e-7a1f-4c3f-9cca-55a87f9a5164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:38.103257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-41e8b52e-7a1f-4c3f-9cca-55a87f9a5164 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:38.104398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-41e8b52e-7a1f-4c3f-9cca-55a87f9a5164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:38.177108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 676/1336] 10.222.0.36 () {58 vars in 1061 bytes} [Thu Jun 16 19:35:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:38.187259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:38.188831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:38.230059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:38.293072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b017d9-f395-421d-9275-bb88293ebfda None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:38.364773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:38.365353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:38.365640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:38.365920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:38.367612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:38.448051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:38.473070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:38.474499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7edec-f05c-4f12-87f9-39c9ffade916 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:38.493236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 661/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:41.676466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:41.677002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:41.677002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:41.678539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 677/1338] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:41.684032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478121-6766-4afe-911c-478b25e9b987 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:41.684423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478121-6766-4afe-911c-478b25e9b987 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:41.684593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478121-6766-4afe-911c-478b25e9b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:41.747352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8b478121-6766-4afe-911c-478b25e9b987 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:41.748554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8b478121-6766-4afe-911c-478b25e9b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:41.845475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 662/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:41.881251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:41.882837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:41.957922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:41.985508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c6e4-1902-42a2-a3e6-8e393c81317e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:42.056760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:42.057401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:42.057685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:42.057962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:42.059871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:42.166198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:42.167651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f17645-7031-42f3-9400-9fec08f13eae service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:42.186025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 678/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:46.549756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:46.550273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:46.550366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:46.551951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 663/1341] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:46.558222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd56450-debb-4855-8aaa-ab5c0d6f7eb2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:46.558222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd56450-debb-4855-8aaa-ab5c0d6f7eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:46.558222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd56450-debb-4855-8aaa-ab5c0d6f7eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:46.574365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:46.638217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6dd56450-debb-4855-8aaa-ab5c0d6f7eb2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:46.639448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6dd56450-debb-4855-8aaa-ab5c0d6f7eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:46.749323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 679/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:46.788098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:46.790239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:46.842565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:46.890602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712157f1-ad0c-4dec-aeef-be395d587263 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:46.894305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:46.894962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:46.895245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:46.895518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:46.905214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:46.906713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c797b-bcf5-40ce-9ea4-f6f6752bb233 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:46.916910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 664/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:50.027598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd7f4-71b1-4084-9aab-89b10d13b130 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:50.028138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd7f4-71b1-4084-9aab-89b10d13b130 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:50.028138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd7f4-71b1-4084-9aab-89b10d13b130 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:50.029513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 680/1344] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:50.034716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884d3b-de20-4d73-971e-87c066f48aaa None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:50.035218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884d3b-de20-4d73-971e-87c066f48aaa None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:50.035321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884d3b-de20-4d73-971e-87c066f48aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:50.062444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5d884d3b-de20-4d73-971e-87c066f48aaa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:50.063670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5d884d3b-de20-4d73-971e-87c066f48aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:50.140026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 665/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:50.177724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd7f4-71b1-4084-9aab-89b10d13b130 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:50.188874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd7f4-71b1-4084-9aab-89b10d13b130 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:50.192440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:50.193165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:50.193514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:50.193858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:50.203268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:50.204752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e86119-ab37-4e51-ac23-37f3aed15f3b service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:50.214564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 681/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:35:54.398886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e7ff-5551-4f55-92e0-f5e4c9a3df1a None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:54.399444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e7ff-5551-4f55-92e0-f5e4c9a3df1a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:54.399538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e7ff-5551-4f55-92e0-f5e4c9a3df1a None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:54.400831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 666/1347] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:54.406083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921303f-3e08-4b62-9d59-0f80e8c9be0e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:54.406542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921303f-3e08-4b62-9d59-0f80e8c9be0e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:54.406772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921303f-3e08-4b62-9d59-0f80e8c9be0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:54.425723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e921303f-3e08-4b62-9d59-0f80e8c9be0e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:54.426858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e921303f-3e08-4b62-9d59-0f80e8c9be0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:54.435679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 682/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:35:54.472296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e7ff-5551-4f55-92e0-f5e4c9a3df1a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:35:54.484103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e7ff-5551-4f55-92e0-f5e4c9a3df1a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:35:54.487675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:35:54.488382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:54.488735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:54.489081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:54.498824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:35:54.500292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b13cfa-4bed-414e-86c4-bf595f08af38 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:35:54.511507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 667/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:35:59.573736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226bad2-baf1-4374-8a49-0a65cfacba43 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:59.574706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226bad2-baf1-4374-8a49-0a65cfacba43 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:59.575208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226bad2-baf1-4374-8a49-0a65cfacba43 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:59.577062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 683/1350] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:35:59.581726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e001bf-1f73-45cf-897f-6b8009875790 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:35:59.582197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e001bf-1f73-45cf-897f-6b8009875790 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:35:59.582321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e001bf-1f73-45cf-897f-6b8009875790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:35:59.652059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d3e001bf-1f73-45cf-897f-6b8009875790 None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:35:59.653695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d3e001bf-1f73-45cf-897f-6b8009875790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:35:59.682286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:35:59.737827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 668/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:03.792781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d98a-eb55-47f9-a891-ec3532099ef4 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:03.793389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d98a-eb55-47f9-a891-ec3532099ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:03.793389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d98a-eb55-47f9-a891-ec3532099ef4 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:03.795002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 684/1352] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:03.800117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ca1-1157-42ff-bdf3-11abcbe7bea8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:03.800588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ca1-1157-42ff-bdf3-11abcbe7bea8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:03.800691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ca1-1157-42ff-bdf3-11abcbe7bea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:03.820380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-47109ca1-1157-42ff-bdf3-11abcbe7bea8 None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:03.821693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-47109ca1-1157-42ff-bdf3-11abcbe7bea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:03.830514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 669/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:36:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:04.426858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=72 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:05.566232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e93ba-963a-4247-8b99-ee16c86ad385 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:05.566892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e93ba-963a-4247-8b99-ee16c86ad385 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:05.566892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e93ba-963a-4247-8b99-ee16c86ad385 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:05.568217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 685/1354] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:05.573301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178e5d-7e9f-4b26-8637-673addd1571f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:05.579043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178e5d-7e9f-4b26-8637-673addd1571f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:05.579043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178e5d-7e9f-4b26-8637-673addd1571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:05.592628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cf178e5d-7e9f-4b26-8637-673addd1571f None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:05.594685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cf178e5d-7e9f-4b26-8637-673addd1571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:05.603925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 670/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:36:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:09.408964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad14831-f33b-4f48-8a2f-b93fb5acddf2 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:09.409535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad14831-f33b-4f48-8a2f-b93fb5acddf2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:09.409535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad14831-f33b-4f48-8a2f-b93fb5acddf2 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:09.410821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 686/1356] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:09.416923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dabdc9-ebeb-4a70-9bfa-b8b74635a296 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:09.417398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dabdc9-ebeb-4a70-9bfa-b8b74635a296 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:09.417482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dabdc9-ebeb-4a70-9bfa-b8b74635a296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:09.436381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-80dabdc9-ebeb-4a70-9bfa-b8b74635a296 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:09.437558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-80dabdc9-ebeb-4a70-9bfa-b8b74635a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:09.445673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 671/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:09.504131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a7d3-a858-42f6-bff2-315a3e6dac45 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:09.504427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a7d3-a858-42f6-bff2-315a3e6dac45 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:09.504708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a7d3-a858-42f6-bff2-315a3e6dac45 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:09.506157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 687/1358] 10.222.0.36 () {54 vars in 961 bytes} [Thu Jun 16 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:09.523749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6bd32-7f8a-43c3-87ca-f88413f44daf None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:09.525501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6bd32-7f8a-43c3-87ca-f88413f44daf None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:09.525501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6bd32-7f8a-43c3-87ca-f88413f44daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:09.588930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c9e6bd32-7f8a-43c3-87ca-f88413f44daf None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:09.590236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c9e6bd32-7f8a-43c3-87ca-f88413f44daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:09.688790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 672/1359] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Jun 16 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:09.696794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45a7d3-a858-42f6-bff2-315a3e6dac45 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:09.711818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45a7d3-a858-42f6-bff2-315a3e6dac45 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:09.718188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:09.718828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:09.719166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:09.719444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:09.728157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:09.729840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506ea98-bda0-4bc8-9bc2-92015e6f3fc5 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:09.739646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 688/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:36:12.088652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1ca8-cfbd-46ca-bb66-f7ba31e888e9 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:12.089252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1ca8-cfbd-46ca-bb66-f7ba31e888e9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:12.089252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1ca8-cfbd-46ca-bb66-f7ba31e888e9 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:12.090958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 673/1361] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:36:12.096774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b3742-3219-41c5-b93e-665b901ef8c4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:12.097360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b3742-3219-41c5-b93e-665b901ef8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:12.097360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b3742-3219-41c5-b93e-665b901ef8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:12.117842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ba3b3742-3219-41c5-b93e-665b901ef8c4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:12.119151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ba3b3742-3219-41c5-b93e-665b901ef8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:12.127795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 689/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:12.174636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac1ca8-cfbd-46ca-bb66-f7ba31e888e9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:12.199647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac1ca8-cfbd-46ca-bb66-f7ba31e888e9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:12.206842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:12.207539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:12.207824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:12.208100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:12.217227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:12.218823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cbc1e-2722-492c-bb85-b804446a0b07 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:12.228878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 674/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:14.818011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b57-a5db-4d4d-9595-f125c6f1a401 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:14.818600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b57-a5db-4d4d-9595-f125c6f1a401 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:14.818600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b57-a5db-4d4d-9595-f125c6f1a401 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:14.820506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 690/1364] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:14.825658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa6b4b-6d60-4bdf-ab10-527e57f58da4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:14.826063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa6b4b-6d60-4bdf-ab10-527e57f58da4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:14.826221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa6b4b-6d60-4bdf-ab10-527e57f58da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:14.845493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6efa6b4b-6d60-4bdf-ab10-527e57f58da4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:14.847168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6efa6b4b-6d60-4bdf-ab10-527e57f58da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:14.857811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 675/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:14.904742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6dc1-37fd-43d8-844e-806657b710bf None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:14.905352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6dc1-37fd-43d8-844e-806657b710bf None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:14.905763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6dc1-37fd-43d8-844e-806657b710bf None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:14.907298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 691/1366] 10.222.0.36 () {54 vars in 961 bytes} [Thu Jun 16 19:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:14.912558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ab30-1d43-4ef3-8ab6-8ede389d744f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:14.912879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ab30-1d43-4ef3-8ab6-8ede389d744f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:14.920356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ab30-1d43-4ef3-8ab6-8ede389d744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:14.931291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-bad5ab30-1d43-4ef3-8ab6-8ede389d744f None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:14.932754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-bad5ab30-1d43-4ef3-8ab6-8ede389d744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:14.944491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 676/1367] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Jun 16 19:36:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:14.952270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6dc1-37fd-43d8-844e-806657b710bf None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:14.973416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6dc1-37fd-43d8-844e-806657b710bf None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:14.982128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:14.982796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:14.983145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:14.983431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:14.993112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:14.994716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f94b7-72cd-4ceb-8e81-46305a04b473 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:15.005205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 692/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:17.368247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322429-a03b-4a7d-9d69-36ba4e218883 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:17.368786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322429-a03b-4a7d-9d69-36ba4e218883 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:17.368786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322429-a03b-4a7d-9d69-36ba4e218883 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:17.370158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 677/1369] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:17.375486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979877-c678-4c94-957e-3ea423b181a8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:17.375889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979877-c678-4c94-957e-3ea423b181a8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:17.376055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979877-c678-4c94-957e-3ea423b181a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:17.393783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-22979877-c678-4c94-957e-3ea423b181a8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:17.395020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-22979877-c678-4c94-957e-3ea423b181a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:17.403507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 693/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:36:17.447093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322429-a03b-4a7d-9d69-36ba4e218883 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:17.462317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322429-a03b-4a7d-9d69-36ba4e218883 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:17.468419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:17.469070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:17.469356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:17.469632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:17.479094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:17.480694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab7f78-fd0d-476c-8854-9e572b809f83 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:17.490704 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 678/1371] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:17.676348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d374b3-50d8-40a4-b391-e611af99075d None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:17.676645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d374b3-50d8-40a4-b391-e611af99075d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:17.676930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d374b3-50d8-40a4-b391-e611af99075d None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:17.678424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 694/1372] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:17.684231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20fd4-7e0d-4886-9a56-47b26dd2f3ee None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:17.684612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20fd4-7e0d-4886-9a56-47b26dd2f3ee None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:17.685011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20fd4-7e0d-4886-9a56-47b26dd2f3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:17.717302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-9cc20fd4-7e0d-4886-9a56-47b26dd2f3ee None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:17.718907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9cc20fd4-7e0d-4886-9a56-47b26dd2f3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:17.727895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 679/1373] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:18.039404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f784118-91f4-4481-a6c7-ef0bc18e217e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:18.039908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f784118-91f4-4481-a6c7-ef0bc18e217e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:18.040348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f784118-91f4-4481-a6c7-ef0bc18e217e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:18.042171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 695/1374] 10.222.0.36 () {54 vars in 963 bytes} [Thu Jun 16 19:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:18.047232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5926ce5-6eb6-4887-87fa-27e2108b3b0c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:18.048100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5926ce5-6eb6-4887-87fa-27e2108b3b0c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:18.048675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5926ce5-6eb6-4887-87fa-27e2108b3b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:18.055865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:18.099919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f5926ce5-6eb6-4887-87fa-27e2108b3b0c None None] MFA Rules not processed for user `56e334de6eee4d5eb8f2fafdb652e9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:18.101294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f5926ce5-6eb6-4887-87fa-27e2108b3b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:18.168249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 680/1375] 10.222.0.36 () {58 vars in 1060 bytes} [Thu Jun 16 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:18.176135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f784118-91f4-4481-a6c7-ef0bc18e217e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:18.191756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f784118-91f4-4481-a6c7-ef0bc18e217e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:18.200326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:18.200982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:18.201277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:18.201695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:18.211751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:18.213193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c817f-0e3d-465c-b503-a90316aa2e4a service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:18.223193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 696/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:19.845365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf8e2-71c9-41c0-adf4-26ad889d37d8 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:19.845906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf8e2-71c9-41c0-adf4-26ad889d37d8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:19.845906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf8e2-71c9-41c0-adf4-26ad889d37d8 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:19.847334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 681/1377] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:19.853337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81833c9-3e77-4cae-bc60-436f480f4fed None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:19.853790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81833c9-3e77-4cae-bc60-436f480f4fed None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:19.853901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81833c9-3e77-4cae-bc60-436f480f4fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:19.871615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e81833c9-3e77-4cae-bc60-436f480f4fed None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:19.872811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e81833c9-3e77-4cae-bc60-436f480f4fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:19.884222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 697/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:19.930408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf8e2-71c9-41c0-adf4-26ad889d37d8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:19.945894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf8e2-71c9-41c0-adf4-26ad889d37d8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:19.951556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:19.952223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:19.952511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:19.952790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:19.962485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:19.964145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ee4c2-a208-4735-8335-41d29c0e3908 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:19.974647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 682/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:22.264499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318fefa-c7c0-4ed7-b445-a5abd085285a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:22.265075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318fefa-c7c0-4ed7-b445-a5abd085285a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:22.265474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318fefa-c7c0-4ed7-b445-a5abd085285a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:22.267125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 698/1380] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:22.272941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412dd7-c7eb-4e2c-b271-86285b581146 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:22.273384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412dd7-c7eb-4e2c-b271-86285b581146 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:22.273480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412dd7-c7eb-4e2c-b271-86285b581146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:22.291568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ce412dd7-c7eb-4e2c-b271-86285b581146 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:22.292750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ce412dd7-c7eb-4e2c-b271-86285b581146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:22.300785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 683/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:22.348355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318fefa-c7c0-4ed7-b445-a5abd085285a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:22.363028 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318fefa-c7c0-4ed7-b445-a5abd085285a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:22.368655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:22.369328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:22.369620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:22.370009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:22.378655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:22.380294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040210eb-ca48-4267-9e5a-509581d78911 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:22.390817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 699/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:24.655006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff73d-6218-42a6-92c8-ad8a43e6c7bf None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:24.655644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff73d-6218-42a6-92c8-ad8a43e6c7bf None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:24.655644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff73d-6218-42a6-92c8-ad8a43e6c7bf None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:24.657081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 684/1383] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:24.662236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a28f4-b3f2-4ac1-af06-0d1643f71b3e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:24.662680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a28f4-b3f2-4ac1-af06-0d1643f71b3e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:24.662791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a28f4-b3f2-4ac1-af06-0d1643f71b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:24.680363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8c0a28f4-b3f2-4ac1-af06-0d1643f71b3e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:24.681488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8c0a28f4-b3f2-4ac1-af06-0d1643f71b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:24.689598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 700/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:24.731248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dff73d-6218-42a6-92c8-ad8a43e6c7bf None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:24.746270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dff73d-6218-42a6-92c8-ad8a43e6c7bf None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:24.751767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:24.752543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:24.752897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:24.753274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:24.762823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:24.764633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853fe6a-6639-441d-9782-8441e90489f2 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:24.776463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 685/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:26.966193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511f01c-5965-4e78-b74b-224bc6df3f41 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:26.966771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511f01c-5965-4e78-b74b-224bc6df3f41 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:26.966873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511f01c-5965-4e78-b74b-224bc6df3f41 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:26.968251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 701/1386] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:36:26.973618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0009-c08f-4dca-9eaa-64d14b38cb6d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:26.974290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0009-c08f-4dca-9eaa-64d14b38cb6d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:26.974290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0009-c08f-4dca-9eaa-64d14b38cb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:26.992246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cd5c0009-c08f-4dca-9eaa-64d14b38cb6d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:26.993487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cd5c0009-c08f-4dca-9eaa-64d14b38cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:27.001745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 686/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:27.047267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511f01c-5965-4e78-b74b-224bc6df3f41 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:27.064768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511f01c-5965-4e78-b74b-224bc6df3f41 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:27.070314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:27.070979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:27.071268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:27.071548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:27.080188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:27.081782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e3d4a-9290-4172-b91b-4d39e303e048 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:27.092176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 702/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:29.446061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc859ff-7e8f-4639-a04c-9cdf8db3463e None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:29.446677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc859ff-7e8f-4639-a04c-9cdf8db3463e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:29.446677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc859ff-7e8f-4639-a04c-9cdf8db3463e None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:29.448083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 687/1389] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:29.453570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b3ba-89e5-432c-94a9-0691c9cee57d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:29.454304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b3ba-89e5-432c-94a9-0691c9cee57d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:29.454304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b3ba-89e5-432c-94a9-0691c9cee57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:29.475336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b398b3ba-89e5-432c-94a9-0691c9cee57d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:29.476558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b398b3ba-89e5-432c-94a9-0691c9cee57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:29.484479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 703/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:36:29.530103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc859ff-7e8f-4639-a04c-9cdf8db3463e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:29.545825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc859ff-7e8f-4639-a04c-9cdf8db3463e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:29.551307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:29.553786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:29.553786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:29.553786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:29.561471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:29.563115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec047be-381b-4de0-afd7-9fc5cbb64dfc service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:29.573117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 688/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:32.568174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7daa-9e24-49c5-a9b8-b371cd6e4249 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:32.568726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7daa-9e24-49c5-a9b8-b371cd6e4249 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:32.568726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7daa-9e24-49c5-a9b8-b371cd6e4249 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:32.570110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 704/1392] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:32.575355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a123-ad42-46e3-b142-312ad8332a58 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:32.575831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a123-ad42-46e3-b142-312ad8332a58 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:32.576016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710a123-ad42-46e3-b142-312ad8332a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:32.594171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1710a123-ad42-46e3-b142-312ad8332a58 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:32.595404 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1710a123-ad42-46e3-b142-312ad8332a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:32.603697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 689/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:32.647061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7daa-9e24-49c5-a9b8-b371cd6e4249 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:32.661282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7daa-9e24-49c5-a9b8-b371cd6e4249 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:32.667247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:32.667881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:32.668167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:32.668444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:32.676432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:32.678074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619109c2-ada3-4791-9773-7546eba87b8e service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:32.687607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 705/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:33.890588 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990426-86a6-4a7c-9403-bfa88a8c9102 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:33.891199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990426-86a6-4a7c-9403-bfa88a8c9102 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:33.891199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990426-86a6-4a7c-9403-bfa88a8c9102 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:33.892648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 690/1395] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:33.897993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a293b-72cf-4b11-af25-ee00c287c0fa None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:33.898411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a293b-72cf-4b11-af25-ee00c287c0fa None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:33.898591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a293b-72cf-4b11-af25-ee00c287c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:33.915805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-522a293b-72cf-4b11-af25-ee00c287c0fa None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:33.917022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-522a293b-72cf-4b11-af25-ee00c287c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:33.924915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 706/1396] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:33.982228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3c38-e201-4b9b-b0e1-51d2502e15cc None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:33.982516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3c38-e201-4b9b-b0e1-51d2502e15cc None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:33.982824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3c38-e201-4b9b-b0e1-51d2502e15cc None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:33.984264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 691/1397] 10.222.0.36 () {54 vars in 962 bytes} [Thu Jun 16 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:33.991608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d5a72-9324-4aac-934c-9e339cb2017a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:33.991904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d5a72-9324-4aac-934c-9e339cb2017a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:33.992191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d5a72-9324-4aac-934c-9e339cb2017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:34.040811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ab3d5a72-9324-4aac-934c-9e339cb2017a None None] MFA Rules not processed for user `20bf88e1a4594d4f973c5dd049aeadd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:34.042031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ab3d5a72-9324-4aac-934c-9e339cb2017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:34.111191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 707/1398] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Jun 16 19:36:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:34.121459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc3c38-e201-4b9b-b0e1-51d2502e15cc None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:34.134160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc3c38-e201-4b9b-b0e1-51d2502e15cc None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:34.137714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:34.138631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:34.138931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:34.139219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:34.171047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:34.171047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1ba0-abdd-4477-9382-765fc1c242cd service glance] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:34.171047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 692/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:35.017089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225f00-2826-406d-b1ac-42965cf12ed8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.017581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225f00-2826-406d-b1ac-42965cf12ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.017663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225f00-2826-406d-b1ac-42965cf12ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.022850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=93 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:35.043351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213f9b5-9639-46f6-a4aa-25ece953a6b2 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.043874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213f9b5-9639-46f6-a4aa-25ece953a6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.043977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213f9b5-9639-46f6-a4aa-25ece953a6b2 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.045255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 693/1400] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:35.050313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.050672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.051104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.070406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d2225f00-2826-406d-b1ac-42965cf12ed8 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:35.071215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:35.071364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d2225f00-2826-406d-b1ac-42965cf12ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:35.072391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:35.080415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 694/1401] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:36:35.126671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:35.139010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dd46c-cf56-4605-a879-cf45d0de55a3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:35.143232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 708/1402] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:36:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:35.144997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:35.145646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.145966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.146251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.155841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:35.157657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b762-0132-4e29-aa9b-24d3eca2edc7 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:35.171219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 695/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:36:35.198599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc72b16-044f-4ff2-ade9-ab7fe95ee496 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.202650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc72b16-044f-4ff2-ade9-ab7fe95ee496 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.202650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc72b16-044f-4ff2-ade9-ab7fe95ee496 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.202650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 709/1404] 10.222.0.36 () {54 vars in 942 bytes} [Thu Jun 16 19:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:35.206843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a2a5-6670-4f3d-98d7-b369e32ccf32 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.207165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a2a5-6670-4f3d-98d7-b369e32ccf32 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.207448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a2a5-6670-4f3d-98d7-b369e32ccf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.258018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-15d9a2a5-6670-4f3d-98d7-b369e32ccf32 None None] MFA Rules not processed for user `e8e2bf373b15496597a6f18d14244495`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:35.259228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-15d9a2a5-6670-4f3d-98d7-b369e32ccf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:35.281410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:35.335759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 696/1405] 10.222.0.36 () {58 vars in 1039 bytes} [Thu Jun 16 19:36:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:36:35.345107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc72b16-044f-4ff2-ade9-ab7fe95ee496 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:35.363261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc72b16-044f-4ff2-ade9-ab7fe95ee496 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:35.373149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:35.373825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:35.374112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:35.374393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:35.383741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:35.385417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705ff47-93ef-467e-9260-ef63c96c1e5c service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:35.394793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 710/1406] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:36:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:36:37.461484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5b769-0864-4592-b1c1-37bdb68d6cd2 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:37.462005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5b769-0864-4592-b1c1-37bdb68d6cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:37.462005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5b769-0864-4592-b1c1-37bdb68d6cd2 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:37.467208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 697/1407] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:36:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:37.468647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961b7cf-a3a9-4479-a480-7720bd649c07 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:37.469101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961b7cf-a3a9-4479-a480-7720bd649c07 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:37.469225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961b7cf-a3a9-4479-a480-7720bd649c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:37.487575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a961b7cf-a3a9-4479-a480-7720bd649c07 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:37.488797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a961b7cf-a3a9-4479-a480-7720bd649c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:37.497564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 711/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:37.543011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5b769-0864-4592-b1c1-37bdb68d6cd2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:36:37.558564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5b769-0864-4592-b1c1-37bdb68d6cd2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:36:37.564071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:36:37.564713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:37.564998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:37.565277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:37.574132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:36:37.575759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c7ecc-fce7-44e6-9fc6-c2e959273352 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:36:37.589310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 698/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:36:47.481577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=48 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:47.584384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=28 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:36:59.095195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f908-9848-48a6-845f-406d25885bb8 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:59.095780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f908-9848-48a6-845f-406d25885bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:59.095780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f908-9848-48a6-845f-406d25885bb8 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:59.097191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 712/1410] 10.222.0.36 () {54 vars in 918 bytes} [Thu Jun 16 19:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:36:59.102750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcd0b-bbb8-4efa-87c9-7c8602cbe0a5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:59.103172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcd0b-bbb8-4efa-87c9-7c8602cbe0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:59.103357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcd0b-bbb8-4efa-87c9-7c8602cbe0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:59.121701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4c7dcd0b-bbb8-4efa-87c9-7c8602cbe0a5 None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:59.122896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4c7dcd0b-bbb8-4efa-87c9-7c8602cbe0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:59.131318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 699/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jun 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:36:59.145519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7878c-da41-424f-9f53-b3d1e4012f43 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:59.145814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7878c-da41-424f-9f53-b3d1e4012f43 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:59.146094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7878c-da41-424f-9f53-b3d1e4012f43 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:59.148127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 713/1412] 10.222.0.36 () {54 vars in 918 bytes} [Thu Jun 16 19:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:36:59.152665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead85d1-de3b-451b-b459-016b583a997c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:36:59.152954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead85d1-de3b-451b-b459-016b583a997c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:36:59.153233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead85d1-de3b-451b-b459-016b583a997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:36:59.171031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1ead85d1-de3b-451b-b459-016b583a997c None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:36:59.172231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1ead85d1-de3b-451b-b459-016b583a997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:36:59.181425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 700/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jun 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:37:01.795237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443be529-60e7-4caf-b417-f53137763e71 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:01.795803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443be529-60e7-4caf-b417-f53137763e71 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:01.795803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443be529-60e7-4caf-b417-f53137763e71 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:01.797170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:01.802275 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae84f8-ad4b-4817-96b7-1f97daf1735c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:01.802722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae84f8-ad4b-4817-96b7-1f97daf1735c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:01.802821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae84f8-ad4b-4817-96b7-1f97daf1735c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:01.821407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cdae84f8-ad4b-4817-96b7-1f97daf1735c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:01.822585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cdae84f8-ad4b-4817-96b7-1f97daf1735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:01.830183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:01.872649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443be529-60e7-4caf-b417-f53137763e71 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:01.886794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443be529-60e7-4caf-b417-f53137763e71 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:01.892196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:01.892831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:01.893118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:01.893401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:01.901862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:01.903554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52b928-2d79-4ec5-9b23-bdf35608f172 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:01.912788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 715/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:02.861776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371123-249f-466f-aeaa-44c05cd6c088 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:02.862352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371123-249f-466f-aeaa-44c05cd6c088 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:02.862352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371123-249f-466f-aeaa-44c05cd6c088 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:02.863712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:02.869154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be2fe-5c58-49f1-b48e-930ae6186e70 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:02.869601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be2fe-5c58-49f1-b48e-930ae6186e70 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:02.869709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be2fe-5c58-49f1-b48e-930ae6186e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:02.887410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fb6be2fe-5c58-49f1-b48e-930ae6186e70 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:02.888612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fb6be2fe-5c58-49f1-b48e-930ae6186e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:02.896399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:03.048939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c371123-249f-466f-aeaa-44c05cd6c088 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:03.063074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c371123-249f-466f-aeaa-44c05cd6c088 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:03.068408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:03.069021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:03.069303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:03.069581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:03.077483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:03.079092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48c9c9-7bf7-4cec-a331-66f0ea133154 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:03.088856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 703/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:04.005187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0bc1d-da47-45ff-86f9-a2c67419294a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:04.005755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0bc1d-da47-45ff-86f9-a2c67419294a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:04.005755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0bc1d-da47-45ff-86f9-a2c67419294a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:04.007186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:04.011953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f031d-d4ef-4d16-a029-2b7bb24cbb36 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:04.012337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f031d-d4ef-4d16-a029-2b7bb24cbb36 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:04.012513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f031d-d4ef-4d16-a029-2b7bb24cbb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:04.030382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-879f031d-d4ef-4d16-a029-2b7bb24cbb36 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:04.031585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-879f031d-d4ef-4d16-a029-2b7bb24cbb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:04.039656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:04.189668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0bc1d-da47-45ff-86f9-a2c67419294a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:04.204139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0bc1d-da47-45ff-86f9-a2c67419294a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:04.211228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:04.211228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:04.211228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:04.211542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:04.219469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:04.221033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6229f0-5065-4c70-9534-1f6911cfd455 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:04.230397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 718/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:05.129314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c86-ebf9-475a-928a-e68f9fb6dc10 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:05.132561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c86-ebf9-475a-928a-e68f9fb6dc10 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:05.132561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c86-ebf9-475a-928a-e68f9fb6dc10 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:05.132561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:05.136238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4089-9e36-4945-afd9-fc951be15f77 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:05.136680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4089-9e36-4945-afd9-fc951be15f77 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:05.136783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e4089-9e36-4945-afd9-fc951be15f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:05.154769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0b5e4089-9e36-4945-afd9-fc951be15f77 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:05.155982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0b5e4089-9e36-4945-afd9-fc951be15f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:05.164168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:05.314624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198c86-ebf9-475a-928a-e68f9fb6dc10 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:05.328628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198c86-ebf9-475a-928a-e68f9fb6dc10 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:05.333969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:05.334591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:05.335425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:05.335707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:05.343829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:05.345519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3631d-9715-4b92-affc-96beb0d3ab8d service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:05.355509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 706/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:06.262076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee2a8-2646-4a99-915b-dc474275b811 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:06.262648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee2a8-2646-4a99-915b-dc474275b811 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:06.262648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee2a8-2646-4a99-915b-dc474275b811 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:06.264044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:06.268808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c42349-5996-4b86-aa4d-62a3088ea767 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:06.269253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c42349-5996-4b86-aa4d-62a3088ea767 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:06.269375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c42349-5996-4b86-aa4d-62a3088ea767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:06.287027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c6c42349-5996-4b86-aa4d-62a3088ea767 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:06.288330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c6c42349-5996-4b86-aa4d-62a3088ea767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:06.296099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:06.448315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ee2a8-2646-4a99-915b-dc474275b811 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:06.462235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ee2a8-2646-4a99-915b-dc474275b811 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:06.467671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:06.468312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:06.468597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:06.468875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:06.477694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:06.479293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a034449-e767-4080-9bc9-7a92a5d229f9 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:06.488490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 721/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:07.391097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292987-b9ce-41b1-8d90-281bdd661618 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:07.391555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292987-b9ce-41b1-8d90-281bdd661618 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:07.391667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292987-b9ce-41b1-8d90-281bdd661618 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:07.393055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:07.397873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94032d3-a73c-4b7d-b0e7-454267bc6fe6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:07.398907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94032d3-a73c-4b7d-b0e7-454267bc6fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:07.399049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94032d3-a73c-4b7d-b0e7-454267bc6fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:07.416935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c94032d3-a73c-4b7d-b0e7-454267bc6fe6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:07.418108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c94032d3-a73c-4b7d-b0e7-454267bc6fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:07.425930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:07.577972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292987-b9ce-41b1-8d90-281bdd661618 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:07.592078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292987-b9ce-41b1-8d90-281bdd661618 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:07.597304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:07.597928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:07.598211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:07.598483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:07.606409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:07.607998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e31b6-6ead-4d5e-a26c-63c9a4489d80 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:07.617497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 709/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:08.555847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c33e-721a-4be7-a1f0-30498ce960f4 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:08.556609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c33e-721a-4be7-a1f0-30498ce960f4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:08.556609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c33e-721a-4be7-a1f0-30498ce960f4 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:08.557791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:37:08.563082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389c2a8-35dc-482e-8bc1-dd3f47ecece3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:08.563504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389c2a8-35dc-482e-8bc1-dd3f47ecece3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:08.563647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389c2a8-35dc-482e-8bc1-dd3f47ecece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:08.581885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e389c2a8-35dc-482e-8bc1-dd3f47ecece3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:08.583121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e389c2a8-35dc-482e-8bc1-dd3f47ecece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:08.591396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:08.745951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316c33e-721a-4be7-a1f0-30498ce960f4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:08.760412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316c33e-721a-4be7-a1f0-30498ce960f4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:08.765741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:08.766373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:08.766661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:08.766966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:08.779223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:08.782033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092bde-f395-4db1-8cca-9f55607dc460 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:08.798819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 724/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:09.713479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee4f-9b1e-4096-bf00-3b9cb0cb1740 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:09.714140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee4f-9b1e-4096-bf00-3b9cb0cb1740 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:09.714140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee4f-9b1e-4096-bf00-3b9cb0cb1740 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:09.715451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:09.720282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36e3a1-02cf-448a-9980-e64bbbda2025 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:09.720710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36e3a1-02cf-448a-9980-e64bbbda2025 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:09.720842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36e3a1-02cf-448a-9980-e64bbbda2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:09.739961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5d36e3a1-02cf-448a-9980-e64bbbda2025 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:09.741258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5d36e3a1-02cf-448a-9980-e64bbbda2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:09.749473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:09.900698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bee4f-9b1e-4096-bf00-3b9cb0cb1740 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:09.915241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bee4f-9b1e-4096-bf00-3b9cb0cb1740 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:09.920499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:09.921126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:09.921412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:09.921693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:09.929583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:09.931195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2f3fb-8a7b-4889-aa2c-810ac0e950d9 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:09.940520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 712/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:10.842590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9621-77b8-4c35-8217-382c384d08d0 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:10.846009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9621-77b8-4c35-8217-382c384d08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:10.846297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9621-77b8-4c35-8217-382c384d08d0 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:10.847733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:37:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:10.853391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf537cb-ff24-415c-97c9-009e3a144253 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:10.853834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf537cb-ff24-415c-97c9-009e3a144253 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:10.853960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf537cb-ff24-415c-97c9-009e3a144253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:10.872673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-adf537cb-ff24-415c-97c9-009e3a144253 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:10.873866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-adf537cb-ff24-415c-97c9-009e3a144253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:10.881592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:11.031943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9621-77b8-4c35-8217-382c384d08d0 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:11.046787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9621-77b8-4c35-8217-382c384d08d0 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:11.052235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:11.055773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:11.055773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:11.055773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:11.062414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:11.064011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6f57-556b-4820-86be-bd256ae4c325 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:11.073217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 727/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:37:13.109206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbaf6-d18f-4f12-9979-8fc45bce0429 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:13.109866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbaf6-d18f-4f12-9979-8fc45bce0429 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:13.109866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccbaf6-d18f-4f12-9979-8fc45bce0429 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:13.111202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:13.116322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c623-80fe-4264-9676-22479d45f93b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:13.116760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c623-80fe-4264-9676-22479d45f93b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:13.116900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c623-80fe-4264-9676-22479d45f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:13.134502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8400c623-80fe-4264-9676-22479d45f93b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:13.135717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8400c623-80fe-4264-9676-22479d45f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:13.143329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:13.181108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccbaf6-d18f-4f12-9979-8fc45bce0429 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:13.191912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccbaf6-d18f-4f12-9979-8fc45bce0429 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:13.195920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:13.196552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:13.196837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:13.197116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:13.205745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:13.207196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00cdd5-1417-471b-a5b0-b7ad12742417 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:13.217030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 715/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:15.572863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:15.573439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:15.573439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:15.574791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:15.580194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e795a-5920-4dad-93aa-c6b0c49f41f9 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:15.580628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e795a-5920-4dad-93aa-c6b0c49f41f9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:15.580791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e795a-5920-4dad-93aa-c6b0c49f41f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:15.598138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6d3e795a-5920-4dad-93aa-c6b0c49f41f9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:15.599375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6d3e795a-5920-4dad-93aa-c6b0c49f41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:15.607593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:15.646809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:15.648503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:15.661130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c7d6f-296a-4207-ac7e-2f7b6bc78688 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:15.665242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:15.665875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:15.666156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:15.666430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:15.675583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:15.677000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0bb02-976a-4c19-89d4-a89c774ef0eb service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:15.687689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 730/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:18.165192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf6251-6733-4b61-bd27-4ef6e4d39fca None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:18.165753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf6251-6733-4b61-bd27-4ef6e4d39fca None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:18.165753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf6251-6733-4b61-bd27-4ef6e4d39fca None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:18.167397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:18.172460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265d88-d9f4-4da7-93fb-53f905993ca5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:18.172842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265d88-d9f4-4da7-93fb-53f905993ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:18.173025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265d88-d9f4-4da7-93fb-53f905993ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:18.191164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4f265d88-d9f4-4da7-93fb-53f905993ca5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:18.192359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4f265d88-d9f4-4da7-93fb-53f905993ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:18.200654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:18.238186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf6251-6733-4b61-bd27-4ef6e4d39fca None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:18.249279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf6251-6733-4b61-bd27-4ef6e4d39fca None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:18.252795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:18.253431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:18.253719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:18.253998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:18.263545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:18.265066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52972f2d-9aef-4bb0-9ad8-8d1b0d03a234 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:18.274224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 718/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:20.488577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae539ac-3051-49f8-b4ec-e14f93ce742c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:20.489132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae539ac-3051-49f8-b4ec-e14f93ce742c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:20.489255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae539ac-3051-49f8-b4ec-e14f93ce742c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:20.490646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:20.496281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aa0d4-15f6-4016-9c1a-d3b7460f9f78 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:20.496685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aa0d4-15f6-4016-9c1a-d3b7460f9f78 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:20.496841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aa0d4-15f6-4016-9c1a-d3b7460f9f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:20.514489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-413aa0d4-15f6-4016-9c1a-d3b7460f9f78 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:20.515702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-413aa0d4-15f6-4016-9c1a-d3b7460f9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:20.523290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:20.560925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae539ac-3051-49f8-b4ec-e14f93ce742c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:20.572007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae539ac-3051-49f8-b4ec-e14f93ce742c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:20.575474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:20.576102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:20.576385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:20.576662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:20.585233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:20.586675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33247680-781d-48e2-81ae-c268a4ea2f73 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:20.595826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 733/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:22.832344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3007b-f4a7-4cd0-a105-96191a341f88 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:22.832923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3007b-f4a7-4cd0-a105-96191a341f88 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:22.832923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3007b-f4a7-4cd0-a105-96191a341f88 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:22.834252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:22.840150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bee3-2193-4791-9928-c854d4163a75 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:22.840563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bee3-2193-4791-9928-c854d4163a75 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:22.840699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bee3-2193-4791-9928-c854d4163a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:22.858888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1ea8bee3-2193-4791-9928-c854d4163a75 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:22.860111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1ea8bee3-2193-4791-9928-c854d4163a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:22.867828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:37:22.905771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3007b-f4a7-4cd0-a105-96191a341f88 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:22.916869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3007b-f4a7-4cd0-a105-96191a341f88 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:22.921411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:22.922065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:22.922342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:22.922615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:22.932121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:22.933539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1368d7-e268-4ed1-a671-a03bea5d17e2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:22.943058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 721/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:25.106498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e98ba-d865-4b19-9b83-461cd515c474 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:25.107071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e98ba-d865-4b19-9b83-461cd515c474 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:25.107071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e98ba-d865-4b19-9b83-461cd515c474 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:25.108445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:25.113587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21634c3-d42b-4753-8f6e-1e42a422d46e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:25.114025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21634c3-d42b-4753-8f6e-1e42a422d46e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:25.114131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21634c3-d42b-4753-8f6e-1e42a422d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:25.131469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a21634c3-d42b-4753-8f6e-1e42a422d46e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:25.132656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a21634c3-d42b-4753-8f6e-1e42a422d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:25.141152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:25.178292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e98ba-d865-4b19-9b83-461cd515c474 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:25.189171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e98ba-d865-4b19-9b83-461cd515c474 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:25.192615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:25.193247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:25.193531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:25.193809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:25.203005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:25.204425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1ba46-54c0-4ab2-948d-c328f80e93b9 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:25.214157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 736/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:27.409892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6214-99ac-491b-a3a1-9dcea2022f30 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:27.410380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6214-99ac-491b-a3a1-9dcea2022f30 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:27.410480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6214-99ac-491b-a3a1-9dcea2022f30 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:27.411871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:27.417499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f342c5-6a3b-40f2-bec2-6739f72e53bf None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:27.417906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f342c5-6a3b-40f2-bec2-6739f72e53bf None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:27.418032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f342c5-6a3b-40f2-bec2-6739f72e53bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:27.436011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c6f342c5-6a3b-40f2-bec2-6739f72e53bf None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:27.437193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c6f342c5-6a3b-40f2-bec2-6739f72e53bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:27.444736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:27.480849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d6214-99ac-491b-a3a1-9dcea2022f30 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:27.491591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d6214-99ac-491b-a3a1-9dcea2022f30 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:27.495658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:27.496285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:27.496567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:27.496843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:27.505529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:27.507018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89db45-682f-496a-991d-9679e0aeb52e service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:27.516227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 724/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:29.703179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c33c1e-6111-42f5-943f-6fe31ef2771c None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:29.703716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c33c1e-6111-42f5-943f-6fe31ef2771c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:29.703848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c33c1e-6111-42f5-943f-6fe31ef2771c None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:29.705171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:29.710491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e7a18-f646-4cd8-8a96-6730b323f064 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:29.710941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e7a18-f646-4cd8-8a96-6730b323f064 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:29.711052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e7a18-f646-4cd8-8a96-6730b323f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:29.722162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=79 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:37:29.744835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a69e7a18-f646-4cd8-8a96-6730b323f064 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:29.746068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a69e7a18-f646-4cd8-8a96-6730b323f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:29.754309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:29.793661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c33c1e-6111-42f5-943f-6fe31ef2771c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:29.799153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=73 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:37:29.804728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c33c1e-6111-42f5-943f-6fe31ef2771c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:29.808342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:29.808979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:29.809263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:29.809812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:29.818552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:29.820016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eebf4-1960-4041-80fd-3c7ad438045e service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:29.829972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 739/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:31.987958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2351278-069c-4cd7-9647-2c67ba4b8d55 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:31.988533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2351278-069c-4cd7-9647-2c67ba4b8d55 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:31.988533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2351278-069c-4cd7-9647-2c67ba4b8d55 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:31.989857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:31.995059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671415-bacb-417d-a3f9-060ec7471a0a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:31.995490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671415-bacb-417d-a3f9-060ec7471a0a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:31.995683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671415-bacb-417d-a3f9-060ec7471a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:32.013617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1d671415-bacb-417d-a3f9-060ec7471a0a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:32.014816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1d671415-bacb-417d-a3f9-060ec7471a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:32.022604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:32.058148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2351278-069c-4cd7-9647-2c67ba4b8d55 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:32.069105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2351278-069c-4cd7-9647-2c67ba4b8d55 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:32.073084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:32.073725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:32.074013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:32.074293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:32.083017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:32.084448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80f3f5-3f64-47a8-b0d1-c488f53f5ce6 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:32.094329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 727/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:34.200847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3b8f-ad9d-40f4-87de-afb0e8126499 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:34.201327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3b8f-ad9d-40f4-87de-afb0e8126499 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:34.201424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3b8f-ad9d-40f4-87de-afb0e8126499 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:34.202757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:34.207901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a0a-911c-4448-bd6e-60007926329b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:34.208324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a0a-911c-4448-bd6e-60007926329b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:34.208491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3a0a-911c-4448-bd6e-60007926329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:34.225763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c96d3a0a-911c-4448-bd6e-60007926329b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:34.226983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c96d3a0a-911c-4448-bd6e-60007926329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:34.235173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:34.271228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3b8f-ad9d-40f4-87de-afb0e8126499 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:34.282265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3b8f-ad9d-40f4-87de-afb0e8126499 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:34.285716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:34.286341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:34.286625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:34.286903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:34.295546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:34.296956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccf1e-fc30-45ec-927c-81e4708ffde4 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:34.306041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 742/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:44.220874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=11 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:37:44.301913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=14 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:37:54.698427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7232-8b5b-4857-a6b4-b3abb7877fc6 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:54.699031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7232-8b5b-4857-a6b4-b3abb7877fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:54.699031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7232-8b5b-4857-a6b4-b3abb7877fc6 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:54.700419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 729/1471] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:54.705706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1770fc-a53f-4816-babf-290cf99828db None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:54.706143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1770fc-a53f-4816-babf-290cf99828db None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:54.706252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1770fc-a53f-4816-babf-290cf99828db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:54.724048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ab1770fc-a53f-4816-babf-290cf99828db None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:54.725227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ab1770fc-a53f-4816-babf-290cf99828db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:54.733499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 743/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:54.779793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf41bb4-113c-4255-a1c0-0d654944abd3 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:54.780083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf41bb4-113c-4255-a1c0-0d654944abd3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:54.780358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf41bb4-113c-4255-a1c0-0d654944abd3 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:54.781696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 730/1473] 10.222.0.36 () {54 vars in 962 bytes} [Thu Jun 16 19:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:54.786439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce36d2-dd00-4151-82da-d8cb4fa43fd7 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:54.786732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce36d2-dd00-4151-82da-d8cb4fa43fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:54.787044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce36d2-dd00-4151-82da-d8cb4fa43fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:54.805573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e7ce36d2-dd00-4151-82da-d8cb4fa43fd7 None None] MFA Rules not processed for user `20bf88e1a4594d4f973c5dd049aeadd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:54.806751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e7ce36d2-dd00-4151-82da-d8cb4fa43fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:54.813335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 744/1474] 10.222.0.36 () {58 vars in 1059 bytes} [Thu Jun 16 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:54.820442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf41bb4-113c-4255-a1c0-0d654944abd3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:54.830770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf41bb4-113c-4255-a1c0-0d654944abd3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:54.834293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:54.834896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:54.835216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:54.835496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:54.844699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:54.846119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf592c-7fec-4ee4-b278-a7ba1f220d8d service glance] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:54.856109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 731/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:37:57.043733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad17f-8d97-478b-b163-282bc81a3a41 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:57.044311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad17f-8d97-478b-b163-282bc81a3a41 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:57.044311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad17f-8d97-478b-b163-282bc81a3a41 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:57.045638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 745/1476] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:37:57.050686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9324-0e50-41cb-86ce-3b85f41dd6c3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:57.051172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9324-0e50-41cb-86ce-3b85f41dd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:57.051268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c9324-0e50-41cb-86ce-3b85f41dd6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:57.069277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-257c9324-0e50-41cb-86ce-3b85f41dd6c3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:57.070466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-257c9324-0e50-41cb-86ce-3b85f41dd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:57.078731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 732/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:57.101516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaad17f-8d97-478b-b163-282bc81a3a41 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:57.114162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaad17f-8d97-478b-b163-282bc81a3a41 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:57.118492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:57.119198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:57.119482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:57.119756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:57.120475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:57.122231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:37:57.122231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:37:57.122231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:37:57.123412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54bc3c-ce95-4289-963f-f3e8ad1cf48f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:57.141862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 746/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Thu Jun 16 19:37:57 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:37:59.291647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bbc2-b90e-474d-9381-e03fbb8c0b0c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:59.292227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bbc2-b90e-474d-9381-e03fbb8c0b0c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:59.292227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bbc2-b90e-474d-9381-e03fbb8c0b0c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:59.293572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 733/1479] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:37:59.299547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b444d-9c2f-4e7f-b9d9-311213575602 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:59.304097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b444d-9c2f-4e7f-b9d9-311213575602 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:59.304097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b444d-9c2f-4e7f-b9d9-311213575602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:59.318222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d28b444d-9c2f-4e7f-b9d9-311213575602 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:37:59.319455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d28b444d-9c2f-4e7f-b9d9-311213575602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:37:59.327568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 747/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:37:59.371017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6bbc2-b90e-474d-9381-e03fbb8c0b0c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:37:59.385031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6bbc2-b90e-474d-9381-e03fbb8c0b0c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:37:59.390536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:37:59.391206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:37:59.391496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:37:59.391773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:37:59.403038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:37:59.403038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385f5d2-3576-495c-9fca-c37de53c1e81 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:37:59.411611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 734/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:38:01.729257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78547-89ba-4b9a-a0e8-478654d2d5c9 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:01.729783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78547-89ba-4b9a-a0e8-478654d2d5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:01.729882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78547-89ba-4b9a-a0e8-478654d2d5c9 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:01.731235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 748/1482] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:38:01.736451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991be0-3601-40f7-8822-faba7a348aae None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:01.736865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991be0-3601-40f7-8822-faba7a348aae None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:01.737029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991be0-3601-40f7-8822-faba7a348aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:01.754645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-90991be0-3601-40f7-8822-faba7a348aae None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:38:01.755885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-90991be0-3601-40f7-8822-faba7a348aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:38:01.764243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 735/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:38:01.799778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e78547-89ba-4b9a-a0e8-478654d2d5c9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:38:01.810807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e78547-89ba-4b9a-a0e8-478654d2d5c9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:38:01.815712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:38:01.820084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:01.820084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:01.820084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:01.826262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:38:01.827715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acad61b-ca60-4741-9aca-bb8463dd0f4d service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:38:01.836928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 749/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:38:04.007860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bdedd-bff1-4863-8b5a-778dd1288f26 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:04.008453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bdedd-bff1-4863-8b5a-778dd1288f26 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:04.008453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bdedd-bff1-4863-8b5a-778dd1288f26 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:04.009780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 736/1485] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:38:04.015085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc1e9-04c9-404a-9279-96dedea1d187 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:04.015503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc1e9-04c9-404a-9279-96dedea1d187 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:04.015650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbc1e9-04c9-404a-9279-96dedea1d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:04.033429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ecbbc1e9-04c9-404a-9279-96dedea1d187 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:38:04.034612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ecbbc1e9-04c9-404a-9279-96dedea1d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:38:04.042888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 750/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:38:04.080083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bdedd-bff1-4863-8b5a-778dd1288f26 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:38:04.091126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bdedd-bff1-4863-8b5a-778dd1288f26 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:38:04.094591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:38:04.095248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:04.095533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:04.095809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:04.105112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:38:04.106530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82b2a9-07f9-4e78-81ba-da7d96b77577 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:38:04.115752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 737/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:38:06.354518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676addd-7740-4bc4-ab70-81d8b31d8b1e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:06.355104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676addd-7740-4bc4-ab70-81d8b31d8b1e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:06.355104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676addd-7740-4bc4-ab70-81d8b31d8b1e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:06.356469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 751/1488] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:38:06.361661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621c8ff-01f2-4c31-b46d-994e69078f78 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:06.362119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621c8ff-01f2-4c31-b46d-994e69078f78 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:06.362224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621c8ff-01f2-4c31-b46d-994e69078f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:06.380030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a621c8ff-01f2-4c31-b46d-994e69078f78 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:38:06.381206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a621c8ff-01f2-4c31-b46d-994e69078f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:38:06.388787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 738/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:38:06.427692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676addd-7740-4bc4-ab70-81d8b31d8b1e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:38:06.438890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676addd-7740-4bc4-ab70-81d8b31d8b1e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:38:06.442350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:38:06.443000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:06.443285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:06.443560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:06.452222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:38:06.453776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a716184-2a5c-48f8-9107-a9bd7df5e2ba service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:38:06.462911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 752/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:38:08.967771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a6c7-291d-4910-924e-d0a65fe7093f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:08.968345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a6c7-291d-4910-924e-d0a65fe7093f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:08.968345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a6c7-291d-4910-924e-d0a65fe7093f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:08.969690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 739/1491] 10.222.0.36 () {54 vars in 926 bytes} [Thu Jun 16 19:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:38:08.974811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254cde7-07ec-4723-99c5-6f76caf6a78d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:08.975272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254cde7-07ec-4723-99c5-6f76caf6a78d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:08.975383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254cde7-07ec-4723-99c5-6f76caf6a78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:08.993341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0254cde7-07ec-4723-99c5-6f76caf6a78d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:38:08.994566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0254cde7-07ec-4723-99c5-6f76caf6a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:38:09.009815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 753/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Thu Jun 16 19:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:38:09.049338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a6c7-291d-4910-924e-d0a65fe7093f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:38:09.059911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a6c7-291d-4910-924e-d0a65fe7093f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:38:09.063722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:38:09.064355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:38:09.064643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:38:09.064924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:38:09.073794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:38:09.075293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0dfdf-c41e-4ebc-8f79-5c7708b3df0a service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:38:09.084488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 740/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:38:18.988325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=33 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:38:19.080434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=33 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:39:42.601084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:42.601694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:42.601694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:42.620231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:39:42.622026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:42.629763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 754/1494] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:39:42.638879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:42.639389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:42.639483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:42.657012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:39:42.658199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:42.665799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 741/1495] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:39:42.670672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:42.680223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e492578-fb8a-46b4-8469-7cf02ef020cd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:42.685272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:42.685923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:42.686211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:42.686492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:42.687240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:39:42.687551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:42.688555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:42.688555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:39:42.688555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:39:42.689432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:42.701038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 755/1496] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:39:42.709228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:42.718273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad382d-f156-4f6f-9d7b-788bc653b7ed None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:42.722753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:42.723422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:42.723705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:42.724003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:42.724698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:42.726475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:42.726475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:39:42.726475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:39:42.727695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:42.865456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 742/1497] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:39:42 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:39:42.869779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:42.879797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa8a6b-7b52-4e8d-8d4b-70db25e36b48 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:42.934176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:42.935253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:42.935677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:42.936109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:42.936811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:42.937828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:42.937828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:39:42.937828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:39:42.939051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.013378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 756/1498] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:39:42 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:39:43.017525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:43.028393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393264e9-a9e7-48e6-b98e-95e07ee56136 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:43.032979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:43.033640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.033930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.034214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.035093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:43.036117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:43.036117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:39:43.036117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:39:43.037002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.050161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 743/1499] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:39:43.054113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:43.063806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908d4a-a34b-471a-8da4-241a2d75991b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:43.068473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:43.069125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.069409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.069682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.070398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:43.071434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:43.071434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:39:43.071434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:39:43.072309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd72f89-116d-468a-95ac-a049d34a3ff0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.085741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 757/1500] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:39:43.089636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:43.099830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d44595-0f52-4920-b609-ae0b8c681afc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:43.104455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:43.105356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.106193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.106474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] PATH_INFO: `/v3/projects/50e706c792d94acbaeb8120d156c134b/users/12b70c635c5f45c194641be5b221d9fc/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.121006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e706c792d94acbaeb8120d156c134b, user_id=12b70c635c5f45c194641be5b221d9fc, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:43.121972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:43.121972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:39:43.121972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:39:43.123936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089acd43-0621-4a54-938f-f0776905cb07 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.191452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 744/1501] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:39:43 2022] PUT /identity/v3/projects/50e706c792d94acbaeb8120d156c134b/users/12b70c635c5f45c194641be5b221d9fc/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:39:43.206649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6b82-65c9-4414-8f33-337df79819f8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.206968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6b82-65c9-4414-8f33-337df79819f8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.207256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6b82-65c9-4414-8f33-337df79819f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.233008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-17cf6b82-65c9-4414-8f33-337df79819f8 None None] MFA Rules not processed for user `12b70c635c5f45c194641be5b221d9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:39:43.234209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-17cf6b82-65c9-4414-8f33-337df79819f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:43.298187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 758/1502] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:39:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:39:43.311712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce1e94-6e7f-4b1b-9455-a2e3ebeaa77a None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.312003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce1e94-6e7f-4b1b-9455-a2e3ebeaa77a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.312282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce1e94-6e7f-4b1b-9455-a2e3ebeaa77a None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.313628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 745/1503] 10.222.0.36 () {54 vars in 963 bytes} [Thu Jun 16 19:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:39:43.318671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaabd4f-2854-4c44-b201-a2c99556f820 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.319002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaabd4f-2854-4c44-b201-a2c99556f820 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.319292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaabd4f-2854-4c44-b201-a2c99556f820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.337453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-abaabd4f-2854-4c44-b201-a2c99556f820 None None] MFA Rules not processed for user `56e334de6eee4d5eb8f2fafdb652e9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:39:43.338759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-abaabd4f-2854-4c44-b201-a2c99556f820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:43.381698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:39:43.389442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 759/1504] 10.222.0.36 () {58 vars in 1060 bytes} [Thu Jun 16 19:39:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:39:43.397233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce1e94-6e7f-4b1b-9455-a2e3ebeaa77a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:43.408214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce1e94-6e7f-4b1b-9455-a2e3ebeaa77a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:43.411764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:43.412448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.412667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.412939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.421262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:43.422685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.431537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 746/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:39:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:39:43.475150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918e4d2-2c3d-42bd-bf84-a4fcd0229294 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.475448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918e4d2-2c3d-42bd-bf84-a4fcd0229294 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.483329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918e4d2-2c3d-42bd-bf84-a4fcd0229294 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.483329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 760/1506] 10.222.0.36 () {52 vars in 829 bytes} [Thu Jun 16 19:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:39:43.483545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:43.541749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc4114-e8fb-471f-a891-cf140c2bcf02 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:43.546261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:43.547157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:43.547441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:43.547714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] PATH_INFO: `/v3/users/12b70c635c5f45c194641be5b221d9fc` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:43.548892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b70c635c5f45c194641be5b221d9fc)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:43.549913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:43.549913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:39:43.549913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:39:43.551182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:43.740080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b70c635c5f45c194641be5b221d9fc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:43.740757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b70c635c5f45c194641be5b221d9fc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:43.755174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b70c635c5f45c194641be5b221d9fc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:44.106389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b70c635c5f45c194641be5b221d9fc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:44.125603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:39:44.128127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b70c635c5f45c194641be5b221d9fc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:44.134318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 747/1507] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:39:43 2022] DELETE /identity/v3/users/12b70c635c5f45c194641be5b221d9fc => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:39:44.139046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:44.139533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:44.139639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:44.157929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:39:44.161047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:44.217861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 761/1508] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:39:44.227501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:44.229763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:44.307639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881455e-187b-4e50-b6c8-05c399b82905 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:44.313120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:44.315707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:44.316008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:44.316555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:44.326394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:44.327903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3712-4ffd-470b-bc32-9b96d62055e2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:44.337436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 748/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:39:44.816740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:39:44.818268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:39:44.830698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529918db-6f1d-4bdc-8f3f-3bf3fc452a6a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:39:44.836046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:39:44.836494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:39:44.836809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:39:44.837058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] PATH_INFO: `/v3/projects/50e706c792d94acbaeb8120d156c134b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:39:44.838242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e706c792d94acbaeb8120d156c134b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:39:44.839254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:39:44.839254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:39:44.839254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:39:44.840551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:39:44.979864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e706c792d94acbaeb8120d156c134b', 'request_id': 'req-fb2707e5-05d7-4523-aecc-043947a8699b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:44.986450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e706c792d94acbaeb8120d156c134b', 'request_id': 'req-fb2707e5-05d7-4523-aecc-043947a8699b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:45.071616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e706c792d94acbaeb8120d156c134b', 'request_id': 'req-fb2707e5-05d7-4523-aecc-043947a8699b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:45.091482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] The token cache is being invalidate because project 50e706c792d94acbaeb8120d156c134b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:39:45.091775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:45.092624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fb2707e5-05d7-4523-aecc-043947a8699b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:39:45.094176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 762/1510] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:39:44 2022] DELETE /identity/v3/projects/50e706c792d94acbaeb8120d156c134b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:39:54.333262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:39:55.087952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:41:28.971525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:28.972127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:28.972127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:28.973469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:28.978200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5590a-d512-4b3f-a3f3-d198ebff8ff2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:28.978636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5590a-d512-4b3f-a3f3-d198ebff8ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:28.978767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5590a-d512-4b3f-a3f3-d198ebff8ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:28.996418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-82c5590a-d512-4b3f-a3f3-d198ebff8ff2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:28.997650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-82c5590a-d512-4b3f-a3f3-d198ebff8ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:29.064542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:29.234187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:29.235918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:29.309560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb12928-bcc2-419b-8414-fdb9cee4f2a4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:29.315572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:29.316249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:29.316539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:29.316824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:29.325139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:29.327360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84aff2-6ebe-4919-b34d-0d25b9b592d3 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:29.336760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 750/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:30.200721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b3cfe-6356-45c5-a0a0-1a0377916501 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:30.201201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b3cfe-6356-45c5-a0a0-1a0377916501 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:30.201311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b3cfe-6356-45c5-a0a0-1a0377916501 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:30.202646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:30.207390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485d45-fb03-49f7-8077-688e530ab7c9 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:30.207812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485d45-fb03-49f7-8077-688e530ab7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:30.207952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485d45-fb03-49f7-8077-688e530ab7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:30.226110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-90485d45-fb03-49f7-8077-688e530ab7c9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:30.227357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-90485d45-fb03-49f7-8077-688e530ab7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:30.235979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:30.388348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b3cfe-6356-45c5-a0a0-1a0377916501 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:30.402882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b3cfe-6356-45c5-a0a0-1a0377916501 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:30.411072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:30.412545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:30.412545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:30.412545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:30.421904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:30.423744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea4af-8ef1-43f0-adbb-969995581c13 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:30.433387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 765/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:30.728212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f2ce-3e80-4cef-8813-f44fe74a2d23 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:30.728525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f2ce-3e80-4cef-8813-f44fe74a2d23 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:30.728808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f2ce-3e80-4cef-8813-f44fe74a2d23 None None] PATH_INFO: `/v3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:30.730168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 752/1517] 10.222.0.23 () {54 vars in 925 bytes} [Thu Jun 16 19:41:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:41:30.743614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6af90-3855-47bb-a30c-05456c52836a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:30.743915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6af90-3855-47bb-a30c-05456c52836a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:30.744203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6af90-3855-47bb-a30c-05456c52836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:30.761786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c0a6af90-3855-47bb-a30c-05456c52836a None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:30.763506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c0a6af90-3855-47bb-a30c-05456c52836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:30.825313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 766/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Jun 16 19:41:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:34.799077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c70ae-0288-483b-8e0c-902d17caa484 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:34.799608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c70ae-0288-483b-8e0c-902d17caa484 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:34.799608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c70ae-0288-483b-8e0c-902d17caa484 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:34.800916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 753/1519] 10.222.0.36 () {54 vars in 960 bytes} [Thu Jun 16 19:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:34.806136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b3112-60ed-42aa-a29b-d4050b151c8b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:34.806679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b3112-60ed-42aa-a29b-d4050b151c8b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:34.806679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b3112-60ed-42aa-a29b-d4050b151c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:34.825810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-656b3112-60ed-42aa-a29b-d4050b151c8b None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:34.827093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-656b3112-60ed-42aa-a29b-d4050b151c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:34.835643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 767/1520] 10.222.0.36 () {58 vars in 1057 bytes} [Thu Jun 16 19:41:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:34.843103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c70ae-0288-483b-8e0c-902d17caa484 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:34.856735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c70ae-0288-483b-8e0c-902d17caa484 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:34.862184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:34.862819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:34.863141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:34.863424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:34.871793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:34.873431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb8f3-182d-4e98-9db0-c8b496d39525 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:34.883672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 754/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:41.699529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc7e56-0b5e-4d77-942c-ba174671bcad None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:41.700099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc7e56-0b5e-4d77-942c-ba174671bcad None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:41.700099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc7e56-0b5e-4d77-942c-ba174671bcad None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:41.701469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:41.706845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95727b-456c-47b5-ac70-7cc410ffe9ab None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:41.707305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95727b-456c-47b5-ac70-7cc410ffe9ab None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:41.707443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95727b-456c-47b5-ac70-7cc410ffe9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:41.725951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cf95727b-456c-47b5-ac70-7cc410ffe9ab None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:41.727353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cf95727b-456c-47b5-ac70-7cc410ffe9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:41.735203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:41.888832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc7e56-0b5e-4d77-942c-ba174671bcad None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:41.903640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc7e56-0b5e-4d77-942c-ba174671bcad None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:41.909669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:41.910306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:41.910608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:41.910878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:41.918961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:41.920591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcf778-22e4-4672-91eb-6f769922efaa service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:41.929870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 769/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:42.793132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8723a2-9167-41e9-b1ef-ea898ed9314c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:42.793715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8723a2-9167-41e9-b1ef-ea898ed9314c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:42.793715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8723a2-9167-41e9-b1ef-ea898ed9314c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:42.795119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 19:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:42.800065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849ba8-6250-41c6-8bc7-5ce2fe93d6f3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:42.800531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849ba8-6250-41c6-8bc7-5ce2fe93d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:42.800643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849ba8-6250-41c6-8bc7-5ce2fe93d6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:42.818712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-09849ba8-6250-41c6-8bc7-5ce2fe93d6f3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:42.820185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-09849ba8-6250-41c6-8bc7-5ce2fe93d6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:42.828005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 19:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:42.981021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8723a2-9167-41e9-b1ef-ea898ed9314c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:42.995387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8723a2-9167-41e9-b1ef-ea898ed9314c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:43.000648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:43.001276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:43.001564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:43.002344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:43.010465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:43.012116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83152e0f-b827-408c-afc1-69717eeadf82 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:43.022113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 757/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:47.142620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:47.143318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:47.143318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:47.144617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:47.149937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c04730-f94e-43d2-b3a9-325026cb3889 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:47.150348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c04730-f94e-43d2-b3a9-325026cb3889 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:47.150507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c04730-f94e-43d2-b3a9-325026cb3889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:47.169140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e9c04730-f94e-43d2-b3a9-325026cb3889 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:47.170369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e9c04730-f94e-43d2-b3a9-325026cb3889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:47.178938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:47.217651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:47.219438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:47.275570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88792f-d106-4034-82a0-6c515417eb70 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:47.279337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:47.280033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:47.280315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:47.280591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:47.289773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:47.291222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99974745-b6ca-4142-a204-9ab44b4fe3f2 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:47.295954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:41:47.301100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 772/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:49.418085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75164a-e560-4d70-a6e3-863cf5467d34 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:49.418628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75164a-e560-4d70-a6e3-863cf5467d34 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:49.418628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75164a-e560-4d70-a6e3-863cf5467d34 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:49.419978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:49.425023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9488b0-2983-42c5-a08f-ec3859f9f0a6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:49.425490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9488b0-2983-42c5-a08f-ec3859f9f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:49.425595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9488b0-2983-42c5-a08f-ec3859f9f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:49.445646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8d9488b0-2983-42c5-a08f-ec3859f9f0a6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:49.446847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8d9488b0-2983-42c5-a08f-ec3859f9f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:49.454543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:49.489604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75164a-e560-4d70-a6e3-863cf5467d34 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:49.500460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75164a-e560-4d70-a6e3-863cf5467d34 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:49.504526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:49.505168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:49.505451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:49.505727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:49.514774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:49.516236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107d7f0-e234-4d7f-be16-54ac4e685812 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:49.526327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 760/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:53.366313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:53.366904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:53.366904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:53.368252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:53.373466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620be2-eadf-4b9d-b111-5f2bc1a6e227 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:53.373885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620be2-eadf-4b9d-b111-5f2bc1a6e227 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:53.374056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620be2-eadf-4b9d-b111-5f2bc1a6e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:53.393988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e3620be2-eadf-4b9d-b111-5f2bc1a6e227 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:53.395220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e3620be2-eadf-4b9d-b111-5f2bc1a6e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:53.406303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:53.680189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:53.681880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:53.738460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b926b80-7d6c-4265-9c6c-a0e1ea46ae1e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:53.741989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:53.742627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:53.742911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:53.743213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:53.751972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:53.753981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e632-fb49-4677-9f7b-c093a94afec9 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:53.763479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 775/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:41:55.905392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:55.905930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:55.905930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:55.907256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 19:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:41:55.912969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97e17-3b12-4ee5-bf32-dfb0709dfa36 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:55.913372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97e17-3b12-4ee5-bf32-dfb0709dfa36 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:55.913537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97e17-3b12-4ee5-bf32-dfb0709dfa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:55.931247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-34c97e17-3b12-4ee5-bf32-dfb0709dfa36 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:41:55.932427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-34c97e17-3b12-4ee5-bf32-dfb0709dfa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:55.940091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 19:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:41:56.203356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:41:56.205078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:41:56.213251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=65 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:41:56.218967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd588a-e7cc-4bb3-b8ae-b0bb9a715bdd None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:41:56.222428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:41:56.223086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:41:56.230386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:41:56.230386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:41:56.232402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:41:56.233823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044db4e-47e8-4d59-b15f-a75650cea2d4 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:41:56.243175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 763/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:42:05.926093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=32 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:42:06.270421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:27.973066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:27.974243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:27.974753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:27.996989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:27.998536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.010717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 777/1540] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.021757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b875d7-5764-4990-817b-70cf0a329ac2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.022967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b875d7-5764-4990-817b-70cf0a329ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.023477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b875d7-5764-4990-817b-70cf0a329ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.045572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d4b875d7-5764-4990-817b-70cf0a329ac2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.047197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d4b875d7-5764-4990-817b-70cf0a329ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.059159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 764/1541] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.064739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.076165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d057573-1b1c-450d-a25a-4d29fc929ded None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.087563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.088615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.089144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.090863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.091424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.091948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.092568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.093837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:28.094379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.096245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efec79-aefc-4384-ad96-1ebbc3df1477 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.125545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 778/1542] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:28.127460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d4e9a-43d2-41ba-8469-2a6ae7394808 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.127989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d4e9a-43d2-41ba-8469-2a6ae7394808 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.128489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d4e9a-43d2-41ba-8469-2a6ae7394808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.131934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.133272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.152723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2a9d4e9a-43d2-41ba-8469-2a6ae7394808 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.154310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2a9d4e9a-43d2-41ba-8469-2a6ae7394808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.156700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 765/1543] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:43:28.157801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.173189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 779/1544] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.176292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51eb1f-1333-4f83-b2d5-2a07db46e32a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.176581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51eb1f-1333-4f83-b2d5-2a07db46e32a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.176702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40d8a0-3f6b-4d92-b968-0a7f91717b6e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.176901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51eb1f-1333-4f83-b2d5-2a07db46e32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.195696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.196384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.196674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.196952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.197667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.200267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c19895-8fbc-4c8d-8ff2-4f940bd126b4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.206568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-be51eb1f-1333-4f83-b2d5-2a07db46e32a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.207838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-be51eb1f-1333-4f83-b2d5-2a07db46e32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.233525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 780/1545] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 19:43:28.234598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e96-57c5-4cf7-8e95-6bd836d93cb0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.234887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e96-57c5-4cf7-8e95-6bd836d93cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.235193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31e96-57c5-4cf7-8e95-6bd836d93cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.267623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-42b31e96-57c5-4cf7-8e95-6bd836d93cb0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.268891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-42b31e96-57c5-4cf7-8e95-6bd836d93cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.278220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 781/1546] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.279622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.279912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.280190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.299806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.301101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.310079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 782/1547] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.311067 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.324483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 766/1548] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:28.329142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824129-86ec-4fce-987b-0ed355f71b93 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.329142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824129-86ec-4fce-987b-0ed355f71b93 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.329142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824129-86ec-4fce-987b-0ed355f71b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.335222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b136ec-296f-4a93-919d-c40ca6b96316 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.366104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fa824129-86ec-4fce-987b-0ed355f71b93 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.367851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fa824129-86ec-4fce-987b-0ed355f71b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.402604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.403613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.404079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.404518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.405246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:28.405555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.407417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.421094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 783/1549] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:28.426529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.437856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2d3e-0658-422e-9671-257d111f77d8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.442991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.443707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.444001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.444262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.445256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:28.445563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.447433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.453113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 767/1550] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.454947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce0c-1fb4-42e9-961d-6a4391443f86 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.455486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce0c-1fb4-42e9-961d-6a4391443f86 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.455993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce0c-1fb4-42e9-961d-6a4391443f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.466068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 784/1551] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:28.469673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.478834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44f239-2d1a-4e5e-88d9-4eb2dfcf5dad admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.484014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.484684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.484970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.485248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.485969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.488223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd1aef-8c98-46a9-b3f3-492762fc88db admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.488372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c523ce0c-1fb4-42e9-961d-6a4391443f86 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.489676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c523ce0c-1fb4-42e9-961d-6a4391443f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.499337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 768/1552] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.500476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.500762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.501039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.519203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.523695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.528242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 769/1553] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.529841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.539102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bf4b2-3c66-4ff9-98fa-a2c39273c367 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.543638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.544280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.544564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.544839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.545508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.547595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55b9de-c90a-489a-802f-24d501b2a85c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.578022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 785/1554] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:28.579332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9f40-fe97-48cb-b228-b6abcc1c9e0b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.579691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9f40-fe97-48cb-b228-b6abcc1c9e0b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.580079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9f40-fe97-48cb-b228-b6abcc1c9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.597530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8ceb9f40-fe97-48cb-b228-b6abcc1c9e0b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.598808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8ceb9f40-fe97-48cb-b228-b6abcc1c9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.606872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 786/1555] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.608069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.608355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.608630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.625949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.627193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.634740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 787/1556] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.635492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.645312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977948-6919-4268-b987-e1ad65d73beb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.649865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.661923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.661923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.661923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.661923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.662356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 770/1557] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:28.662464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:28.662464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:43:28.662464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:43:28.662743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.662840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dbc35-1b9a-4c11-819c-b1a75d718a02 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.662957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.662957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.674624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.675888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.733215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 771/1558] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.734095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.743280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52885e-9497-4656-ba35-6e4524e53002 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.748671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.749423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 788/1559] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:28.749560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.749657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.749993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.750635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344116d1-6850-4106-818a-4563238c832f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.750768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:28.750951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344116d1-6850-4106-818a-4563238c832f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.751074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.751259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344116d1-6850-4106-818a-4563238c832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.752481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:28.752481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:43:28.752481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:43:28.753056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.772547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 772/1560] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:28.772547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.772816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-344116d1-6850-4106-818a-4563238c832f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.772816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-344116d1-6850-4106-818a-4563238c832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.776063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c952f-48a2-403c-9acf-e8ab95f58171 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.788211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:28.810584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:28.831788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 789/1561] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.832595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344116d1-6850-4106-818a-4563238c832f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.833597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.834269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.834556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.834837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.835633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.837234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bce0a-854b-42d1-ac2f-eeab35c37687 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.841763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344116d1-6850-4106-818a-4563238c832f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.846389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.849906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.850055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 773/1562] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:28.850153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.850153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.850316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7881-dd46-48d1-aee6-e1cf6419c8d0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.850407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:28.850494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7881-dd46-48d1-aee6-e1cf6419c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.850570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.850656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7881-dd46-48d1-aee6-e1cf6419c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.850743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.862981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 790/1563] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:28.863880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.867505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5e0c7881-dd46-48d1-aee6-e1cf6419c8d0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.868756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5e0c7881-dd46-48d1-aee6-e1cf6419c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.874441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272bc6-260a-4143-97d5-13c342726a32 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.878038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 774/1564] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.879026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.879464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.879677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.879803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.879985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.880087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.880238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.881014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.882914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:28.898410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:28.899899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:28.907932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 775/1565] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:28.908676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:28.918253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c3508-7b3e-4fcf-a82a-de347e7a0951 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:28.922782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:28.923440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:28.923729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:28.924013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:28.924691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:28.925707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:28.925707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:43:28.925707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:43:28.926902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.002857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 791/1566] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.002857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.012770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd46038-17be-4113-86cc-653b5f62b61a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.017615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.018283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.018571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.018852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.019572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.021496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.025764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 776/1567] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.025764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.035654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b84c-5281-4317-bacd-684c1b546240 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.040530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.041634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.041928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.042213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.042971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:29.043292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.044900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.056608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 777/1568] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:29.057384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.066672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e2fca-a0b6-4443-aa4f-0370fa8ecc47 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.071419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.072044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.072332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.072627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.073338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.075408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.084949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 778/1569] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.085686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.095287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff537-e78f-4774-be2a-1e0f6db4eb05 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.100216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 792/1570] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:28 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.100216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.100428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.104312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.104312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.104312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.104312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.104312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.163340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c433e06-9ce2-431c-9f2e-5cf6e51563f4 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.168533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.169212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.169501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.169800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.170521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:29.170829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.172467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.178467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 779/1571] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.179563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.189634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 793/1572] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:29.190540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.208105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d72ab-117f-4ce1-b518-efe44abfef4e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.244150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83c7b6-05fc-460c-a637-4287988539af admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.249317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.250013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.250297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.250577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.251333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:29.251640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.253305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.264833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 780/1573] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:29.274126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.274126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.274126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.274126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.274511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.274597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.274597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.277653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3e16-4ce9-434a-a146-62b6de91d21a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.280622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 794/1574] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.281333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.282439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.283122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.283474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.283727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.284442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.286123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.290759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f34b6-10cb-4c75-9443-293d5945ac9f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.296174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.296822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.297109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.297243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 781/1575] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.297390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.297900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.298069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.299984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.309917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1984-1374-457a-ae45-dee6592553da admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.314605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.315497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.315782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.316052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] PATH_INFO: `/v3/projects/38629fbda01748eb98e2dca20a3da10a/users/8b5a188cee0d4727a14153088b8d95c6/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.330849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38629fbda01748eb98e2dca20a3da10a, user_id=8b5a188cee0d4727a14153088b8d95c6, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.333509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.336485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:29.411329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:29.418260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 795/1576] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.419202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.446227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 782/1577] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:29 2022] PUT /identity/v3/projects/38629fbda01748eb98e2dca20a3da10a/users/8b5a188cee0d4727a14153088b8d95c6/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:29.447078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.483620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e226-4945-439d-9f03-1ac6211c049c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.511699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b26ad6-1591-47e8-ba6a-271ebc27c462 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.516986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.525639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.525639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.525639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.525639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.525639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.540032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.540734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.541018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.541293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.541974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.544450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.644758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 783/1579] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.644974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 796/1579] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.645087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.645194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.651527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5d5a5-f245-471e-bf9b-60786b6a17d0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.652510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acafd6-7d71-4c3b-9c95-06ff03451e4e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.656872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.657085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.657547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.657727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.657974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.658131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.658321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.658428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.658957 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.659082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.660849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.660977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.732262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 784/1580] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.733081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.747391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 797/1581] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.747391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.794581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc31f-7449-4d8f-90ba-974f54a77977 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.807683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f78e-dd5c-4b30-9e3b-6d95773843be admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.812742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.813406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.813697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.813980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.814711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.816442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.826313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 798/1582] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.827115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.836374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a6925-8149-43a2-b706-d0cfa27ca51a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.841151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.841788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.842073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.842351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.843052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.844916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.849843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.850506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.850793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.851113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.851848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.852495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:29.853440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ef77f-3893-433a-99be-7e38c65d9535 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.863889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 785/1583] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.865170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.865408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.865690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.877531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:29.882570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:29.883923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:29.891579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 786/1584] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:29.892397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.901747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef3d05-e4c5-4ff5-832c-9ab5f88dfdee None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.906159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.906792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.907108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.907387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.908097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.909697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.919703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 787/1585] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.920583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.929632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdfa32-a3b3-482b-938b-01cf20a355c6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.934574 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.935205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.935489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.935763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.936448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.937955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.947451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 788/1586] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:29.948343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.955547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 799/1587] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:29.955547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:29.960218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a63-5618-4b4c-98d0-4b07e55789d9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.962669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.963342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.963631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.963906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.965129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.966466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdbf4-2514-431d-be8e-71dd12224e36 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:29.967041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:29.971085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:29.971737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:29.972025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:29.972298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:29.972978 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:29.974901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.074379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 789/1588] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:30.075174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.075492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 800/1589] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:29 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:30.076770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.085357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df3e26-1c55-4e26-84ad-504448674f29 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.086494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd08c-2593-4c61-90cb-f6622cd08b81 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.090570 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.091595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.091814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.091980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.092313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] PATH_INFO: `/v3/projects/83a154e7e1db49adb3d342788331e0e5/users/d1dca219a643470590ab0325e705e037/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.092824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.093176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.093420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] PATH_INFO: `/v3/projects/47a4ab888dd74218ab5d7812ad095371/users/4df05cbf303c4524a665b429f240a17a/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.111213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a4ab888dd74218ab5d7812ad095371, user_id=4df05cbf303c4524a665b429f240a17a, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.112564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a154e7e1db49adb3d342788331e0e5, user_id=d1dca219a643470590ab0325e705e037, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.113717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:30.113717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:43:30.113717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:43:30.115850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.116918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.196670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 801/1590] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/47a4ab888dd74218ab5d7812ad095371/users/4df05cbf303c4524a665b429f240a17a/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:30.196670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.277244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 790/1591] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/83a154e7e1db49adb3d342788331e0e5/users/d1dca219a643470590ab0325e705e037/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:30.277244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.277440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb952b-978f-46d9-ad1a-a4fc28c764d0 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.303509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.304167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.304433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.304708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.306276 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1539-8bcc-46bf-bdac-947c301d5f87 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.311798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.312463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.312751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.313030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.313791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.314318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.315435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.315760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.325760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 802/1592] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:30.326033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 791/1593] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.326954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.330078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.341277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed34e8a-7821-4ba4-99b4-e9eaa5e41cd0 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.344931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcb47-0cde-4bfe-9a7a-ffc58c21390b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.346460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.347212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.347502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.347781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.348507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.350184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.350478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.351342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.351630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.351910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.352637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.354213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.361702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 803/1594] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.362740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.371168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 792/1595] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.371168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.375407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfce38-32ba-43c2-8c93-85cf06cd1b67 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.379898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3897cf-1857-45d5-bd98-423ee40ec2a2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.380364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.381026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.381376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.381657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.382378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.384001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb8a36-2583-4829-954b-1fd14855acf5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.385461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.386836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.387220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.387496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.388247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.390505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008445-c4e5-4d89-9e04-e205947ecf60 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.396511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 804/1596] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.398117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.402751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.402751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.403009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 793/1597] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.403009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.403009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.403338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.422236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:30.423503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:30.430039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] MFA Rules not processed for user `d1dca219a643470590ab0325e705e037`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:30.431499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:30.431637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 805/1598] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:30.432119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.441816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f37d-ff7e-47eb-a56f-cd6f32a63f86 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.447270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.448197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.448509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.449291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] PATH_INFO: `/v3/projects/5544755ae458477d88d411b080a2517a/users/d9320ca5b2a9499a86d5614b55eddede/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.465961 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5544755ae458477d88d411b080a2517a, user_id=d9320ca5b2a9499a86d5614b55eddede, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.468882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.497131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:30.500889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 794/1599] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:30.501682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.513945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e81ea6-0d9c-4dfd-8e1a-2f99db9ee5da None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.519087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.529069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.529069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.529069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] PATH_INFO: `/v3/projects/9f531d0fe3d64ff6ba4d2e960f5ee84a/users/192c8d39e40349c7b595f1a423d7a521/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.535620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f531d0fe3d64ff6ba4d2e960f5ee84a, user_id=192c8d39e40349c7b595f1a423d7a521, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.538261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.577562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 806/1600] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/5544755ae458477d88d411b080a2517a/users/d9320ca5b2a9499a86d5614b55eddede/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:30.578948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.596886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:30.638727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5c0-2d3f-42c5-9f75-cf6e9279e9d5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.643275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.643938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.644223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.644500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.645214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.646788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.660295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 807/1601] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.660295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.665646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 795/1602] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/9f531d0fe3d64ff6ba4d2e960f5ee84a/users/192c8d39e40349c7b595f1a423d7a521/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:30.666470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.668840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b986-023c-42a3-9f97-5de145e35bfb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.741577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.742537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca8bf-ecac-4ed5-b9f0-75247421ca6d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.742728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.743139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.743509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.744458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.746711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.749779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.750461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.750748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.751183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.751931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.754093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.758077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 808/1603] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.758804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.760617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:30.767198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 796/1604] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:30.768106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.779253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f596f-4d4e-4594-becb-26cf2d8ca3b7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.784103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.799012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.799012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.799012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.799012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.799012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90521e-ead7-41b7-8bf5-6c5f78584ade admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.821475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565e214-ad42-4065-9ef2-da46f375d383 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.830839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.830839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.830839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.830839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.835552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.837023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c05f6-7338-4f0d-bdb0-90092f62075d service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.846344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 809/1605] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:30.847534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.847821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.848099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.865155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:30.866441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:30.874896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 810/1606] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:30.875733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:30.885212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc1e67-796a-4ab3-bb2f-314a90d551b2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:30.890383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:30.891297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.891581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.891885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] PATH_INFO: `/v3/projects/df6beb24a9cc41dba4182b369e045f1a/users/2183fd35fd7c412cb5bbdfa24a0c91db/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.908605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6beb24a9cc41dba4182b369e045f1a, user_id=2183fd35fd7c412cb5bbdfa24a0c91db, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:30.914724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 797/1607] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:30 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:30.914853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:30.915015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:30.915015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:30.915015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:30.941771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] MFA Rules not processed for user `192c8d39e40349c7b595f1a423d7a521`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:30.943101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:30.989878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 811/1608] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/df6beb24a9cc41dba4182b369e045f1a/users/2183fd35fd7c412cb5bbdfa24a0c91db/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:30.990752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.029988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 798/1609] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:31.031371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.036653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:31.094350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804edf-9327-4d8d-9b1b-7350502840cb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.100095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.109225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.109225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.109225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] PATH_INFO: `/v3/projects/19487af2cbd5445d88ca0df1e308ccc3/users/eb090bd3ce3741268abd614d943d21fb/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.109714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:31.117965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19487af2cbd5445d88ca0df1e308ccc3, user_id=eb090bd3ce3741268abd614d943d21fb, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.121256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.130453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80e655-3c54-4e39-99bf-184b17ad2bc1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.136137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.137214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.137363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.137584 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] PATH_INFO: `/v3/projects/7153c09b2d0b4805ad0360afb7ecd3ca/users/2d0d5723417d4b60bc51e47f28d09902/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.155196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7153c09b2d0b4805ad0360afb7ecd3ca, user_id=2d0d5723417d4b60bc51e47f28d09902, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.158463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.258715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 799/1611] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:31 2022] PUT /identity/v3/projects/7153c09b2d0b4805ad0360afb7ecd3ca/users/2d0d5723417d4b60bc51e47f28d09902/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:31.264108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.268116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 812/1611] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:30 2022] PUT /identity/v3/projects/19487af2cbd5445d88ca0df1e308ccc3/users/eb090bd3ce3741268abd614d943d21fb/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:31.269216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.326650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907b3b0-e076-4490-88ef-3243a94e8464 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.330471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.331159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.331453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.331556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065adfd-a64c-4fcb-94d3-8bdc1f0a1448 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.331737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.402220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.403203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.403732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.404094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.405109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.407210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b728b2-3ba9-4df0-a6ab-ab659f9f4150 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.416224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.416224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe840b5-4a15-4af9-8918-0aa9ffbf51c2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.426975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 800/1612] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:31.432901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c33c-4b5d-43cd-9d5a-6fd29c0190b8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.434332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c33c-4b5d-43cd-9d5a-6fd29c0190b8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.434714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c33c-4b5d-43cd-9d5a-6fd29c0190b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.469495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-9065c33c-4b5d-43cd-9d5a-6fd29c0190b8 None None] MFA Rules not processed for user `2183fd35fd7c412cb5bbdfa24a0c91db`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:31.471069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9065c33c-4b5d-43cd-9d5a-6fd29c0190b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:31.503890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 813/1613] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:31.505274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dac62-a554-4c1c-8b0a-cc71f33307e8 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.505564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dac62-a554-4c1c-8b0a-cc71f33307e8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.505843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dac62-a554-4c1c-8b0a-cc71f33307e8 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.507485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 814/1614] 10.222.0.36 () {54 vars in 961 bytes} [Thu Jun 16 19:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:31.508521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477785e-37af-4715-a6ac-760502b5521e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.508804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477785e-37af-4715-a6ac-760502b5521e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.509082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477785e-37af-4715-a6ac-760502b5521e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.548130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e477785e-37af-4715-a6ac-760502b5521e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:31.549383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e477785e-37af-4715-a6ac-760502b5521e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:31.551440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 801/1615] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:31.554078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.554078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.554309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.561908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 815/1616] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:31.562692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477785e-37af-4715-a6ac-760502b5521e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.573482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477785e-37af-4715-a6ac-760502b5521e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.578259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.578986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.579246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.579521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.580285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.581893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9623c3b-64d5-496b-89c4-81b4870ab9dc admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.593044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] MFA Rules not processed for user `eb090bd3ce3741268abd614d943d21fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:31.594271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:31.597778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 816/1617] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:31.597778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.597778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.597778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.629042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:31.629042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:31.663475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:31.686888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:31.698226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 802/1618] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:31.704598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.721616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525123b-7aef-4139-91a0-9168ef931474 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.729684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.731100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.731499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.732159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.732865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.736176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.795374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 817/1619] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:31.798999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.811717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fa4c0-204e-40ee-ba1e-9bd39253e28d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.819111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.819111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.819111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.819111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.847665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.852017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 803/1620] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:31 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:31.852174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f80ac2-c7b2-455d-89d6-d669207b86a1 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:31.852288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.864189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 818/1621] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:31.865790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa935-3d0f-4fbd-9639-dd452d4729d8 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.866192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa935-3d0f-4fbd-9639-dd452d4729d8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.866701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa935-3d0f-4fbd-9639-dd452d4729d8 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.870533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 819/1622] 10.222.0.36 () {54 vars in 961 bytes} [Thu Jun 16 19:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:31.870533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa935-3d0f-4fbd-9639-dd452d4729d8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:31.949974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb823-1eab-4244-a6ef-c4fc1fd7b600 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.957029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:31.958272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:31.958740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:31.959769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] PATH_INFO: `/v3/projects/9bd1bc122ce843c1b54a632993f24bc7/users/5223ce69f5604973b55299f794375971/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:31.990765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1bc122ce843c1b54a632993f24bc7, user_id=5223ce69f5604973b55299f794375971, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:31.991763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa935-3d0f-4fbd-9639-dd452d4729d8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:31.995197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.017268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.017268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.017268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.017268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.074129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 804/1623] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:31 2022] PUT /identity/v3/projects/9bd1bc122ce843c1b54a632993f24bc7/users/5223ce69f5604973b55299f794375971/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:32.074129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.120106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.122369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ea5a-6e4a-48bb-93fa-a4a05ce3d68a service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.153413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 820/1624] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:32.159976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.164761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.165184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.179143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36adbd-04ed-4785-87c5-4fbd26466905 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.186235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.187070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.187476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.187833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.188552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.191087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.207347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:32.208840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:32.289261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 805/1625] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:32 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:32.289261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.303801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 821/1626] 10.222.0.36 () {58 vars in 1058 bytes} [Thu Jun 16 19:43:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:32.307760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.314813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65684744-c09e-4d98-91c9-7f9fa17eb1ec admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.320604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.321663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.321663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.321861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.322564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.324192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c0d2d-0ddf-4c24-8582-28c6bd0dfe7d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.324723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.329683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.330619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.331256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.331726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.332970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.336624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.341107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 806/1627] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:32.341107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.347512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:32.357954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8eb67-1698-4270-90ce-a5343f4e3b07 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.364377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.365163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.365518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.365930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.376245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 822/1628] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:32.376245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.396066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dc290-3006-40e6-a3cc-ba57813b525f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.423267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.424377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.424707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.424993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] PATH_INFO: `/v3/projects/9bd1bc122ce843c1b54a632993f24bc7/users/5223ce69f5604973b55299f794375971/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.427471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1bc122ce843c1b54a632993f24bc7, user_id=5223ce69f5604973b55299f794375971, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.430688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.458987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 823/1629] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:32 2022] PUT /identity/v3/projects/9bd1bc122ce843c1b54a632993f24bc7/users/5223ce69f5604973b55299f794375971/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:32.458987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.494960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.497955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2008a56-b387-4429-baf1-4af5195ea39b service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.522405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 807/1630] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:32.524068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e1c2-10b0-42a1-a629-41e81fa34cf2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.524478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e1c2-10b0-42a1-a629-41e81fa34cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.524839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e1c2-10b0-42a1-a629-41e81fa34cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.534022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:32.560626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cc64e1c2-10b0-42a1-a629-41e81fa34cf2 None None] MFA Rules not processed for user `5223ce69f5604973b55299f794375971`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:32.562662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cc64e1c2-10b0-42a1-a629-41e81fa34cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:32.567481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8607-f5e4-4c3d-af0f-bd0919486db8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.577658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.582199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.584209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.584209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.584209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.591530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d526d-d771-431d-a482-d9e34b5d1669 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.619545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 824/1631] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:32.619545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37444b-eaf2-44de-9704-b9209334470e None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.619545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37444b-eaf2-44de-9704-b9209334470e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.619545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37444b-eaf2-44de-9704-b9209334470e None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.621394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 825/1632] 10.222.0.36 () {54 vars in 912 bytes} [Thu Jun 16 19:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:32.622577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37444b-eaf2-44de-9704-b9209334470e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.642043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37444b-eaf2-44de-9704-b9209334470e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.653758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.656447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.656965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.658857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] PATH_INFO: `/v3/projects/31b8a8b5ec064e2880d21b147ea04589/users/e08d385b94cd44f1af9a54f349d74731/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.695745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 808/1633] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:32.699181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344d232-c9d0-4abd-9e5c-4c2990718e6d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.699486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344d232-c9d0-4abd-9e5c-4c2990718e6d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.699773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344d232-c9d0-4abd-9e5c-4c2990718e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.705152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b8a8b5ec064e2880d21b147ea04589, user_id=e08d385b94cd44f1af9a54f349d74731, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:32.707625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:32.741536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8344d232-c9d0-4abd-9e5c-4c2990718e6d None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:32.744737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8344d232-c9d0-4abd-9e5c-4c2990718e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:32.791230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 826/1634] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:32 2022] PUT /identity/v3/projects/31b8a8b5ec064e2880d21b147ea04589/users/e08d385b94cd44f1af9a54f349d74731/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:32.792088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:32.876361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 809/1635] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:43:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:32.877182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.877687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.878028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:32.929636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] MFA Rules not processed for user `e08d385b94cd44f1af9a54f349d74731`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:32.929806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1ee3-88ea-48ef-b8dc-25ebe93f5435 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:32.929920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:32.938215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:32.938899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:32.939231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:32.939513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.002830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:33.017350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 810/1636] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:33.018369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.029740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf671d-4432-4a15-9b2a-3a42345debb3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.040258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.040980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.040980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.040980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.056477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.060150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e51724-e35b-426d-a091-c83e9ab65049 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.065561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:33.098620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 827/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:33.101488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc742-2666-4535-9796-18126a82157a None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.102161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc742-2666-4535-9796-18126a82157a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.105512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc742-2666-4535-9796-18126a82157a None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.113834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 828/1638] 10.222.0.36 () {54 vars in 912 bytes} [Thu Jun 16 19:43:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:33.117753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc742-2666-4535-9796-18126a82157a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.186248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.188134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7db928-c022-47ed-be81-b54d92b50ea3 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.203946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 811/1639] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:33.208509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.208509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.208509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.227427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:33.228861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:33.247747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 812/1640] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:43:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:33.283414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc742-2666-4535-9796-18126a82157a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.290114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.291264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.291715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.292138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.293023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.295274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.377514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.405831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c317637-fab7-434c-8a21-ecccda26e2a1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.411792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.412944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.414023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.414726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.445490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 829/1641] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:33 2022] POST /identity/v3/projects => generated 385 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:33.445598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.449407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.469573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.505477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 813/1642] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:43:33.506733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.532199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9b575-423a-4247-8734-23def2c22c9f service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.599423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.600309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.600606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.600946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.601870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c8282-c791-4f22-9ad2-d5aaabc1dd0b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.621600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.622539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.622894 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.623270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.624039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.628370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.748864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:33.760456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 830/1643] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:33 2022] POST /identity/v3/users => generated 299 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:33.761534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.773462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.776085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.786695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e40ad-df0b-4e84-b946-98fc62ff129f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.937797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.937797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.937797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.937797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:33.938888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:33.938888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:33.939820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 814/1644] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:33.939924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 831/1645] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:33.940023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:33.940023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044a95-921b-4e36-9697-963e39609a0d service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:33.940023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:33.940023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:33.940371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:33.940371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] PATH_INFO: `/v3/projects/7b8120e381004f30bbc04c17af352a33/users/93b6a5e7e2484b3ba4e8789f0dacecf9/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:34.137533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8120e381004f30bbc04c17af352a33, user_id=93b6a5e7e2484b3ba4e8789f0dacecf9, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:34.141002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:34.259349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 815/1646] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:33 2022] PUT /identity/v3/projects/7b8120e381004f30bbc04c17af352a33/users/93b6a5e7e2484b3ba4e8789f0dacecf9/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:43:34.263580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:34.299099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652cf2d-b18d-4e12-9d19-6135bb8090f9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:34.350082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:34.389330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:34.390376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:34.390889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:34.392085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:34.394880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:34.397020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba589-90a0-45e1-aa54-e854be9daf17 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:34.413132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:34.428182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 832/1647] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:34.443419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ec30d-72f1-423c-bbc0-af38abecc59a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:34.461653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:34.463375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:34.463658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:34.463938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] PATH_INFO: `/v3/projects/7b8120e381004f30bbc04c17af352a33/users/93b6a5e7e2484b3ba4e8789f0dacecf9/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:34.467409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8120e381004f30bbc04c17af352a33, user_id=93b6a5e7e2484b3ba4e8789f0dacecf9, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:34.471184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:34.509090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 816/1648] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:34 2022] PUT /identity/v3/projects/7b8120e381004f30bbc04c17af352a33/users/93b6a5e7e2484b3ba4e8789f0dacecf9/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:34.511621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:34.512122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:34.513196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:34.611359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] MFA Rules not processed for user `93b6a5e7e2484b3ba4e8789f0dacecf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:34.613001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:34.874236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 833/1649] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:34.876982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:34.887036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:35.018519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d983d-d281-4f16-b83e-e7043d0c1fba None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:35.027353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:35.028631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:35.028631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:35.028839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:35.053643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:35.061243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd330-394f-4a6a-a53b-e0b0f704701e service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:35.096698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 834/1650] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:35.121867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3687844-6ca4-425c-8c9f-f5044a1e6f4c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:35.134673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:35.135789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:35.136115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:35.136536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] PATH_INFO: `/v3/users/192c8d39e40349c7b595f1a423d7a521` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:35.139571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192c8d39e40349c7b595f1a423d7a521)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:35.144333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:35.488572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192c8d39e40349c7b595f1a423d7a521'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:35.496626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192c8d39e40349c7b595f1a423d7a521'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:35.514443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192c8d39e40349c7b595f1a423d7a521'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:35.524880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:35.629353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192c8d39e40349c7b595f1a423d7a521'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:35.657070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192c8d39e40349c7b595f1a423d7a521'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:35.693890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:35.694387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:35.694467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:35.708371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 817/1651] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:34 2022] DELETE /identity/v3/users/192c8d39e40349c7b595f1a423d7a521 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:35.733218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:35.734535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:35.892116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:35.915150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 835/1652] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:43:35.927519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:35.929052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:36.066367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4341-6f02-4864-829b-9a373adb95c9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:36.074182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:36.074881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:36.075273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:36.075557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:36.109436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:36.111586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:36.139909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 818/1653] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:36.948397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:36.952607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:37.027647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b2d64-fcf7-4261-a99f-b5dc74707769 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:37.042557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:37.044901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:37.052311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:37.055243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.055542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.055828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.064951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:37.089106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779aedcf-be0e-406d-b941-fa6912247018 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:37.108673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:37.110125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.110421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.110756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] PATH_INFO: `/v3/projects/9f531d0fe3d64ff6ba4d2e960f5ee84a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.113866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f531d0fe3d64ff6ba4d2e960f5ee84a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:37.115069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: WARNING py.warnings [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:37.115069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: warnings.warn(msg) Jun 16 19:43:37.115069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: Jun 16 19:43:37.118085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:37.268649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:37.272328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd66203-093f-4b90-98c9-0b5e3e4d68ea service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:37.320714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 836/1654] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:37.333329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f531d0fe3d64ff6ba4d2e960f5ee84a', 'request_id': 'req-1e9924f6-57cb-4005-84f0-1c121f0a0d89'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:37.357248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f531d0fe3d64ff6ba4d2e960f5ee84a', 'request_id': 'req-1e9924f6-57cb-4005-84f0-1c121f0a0d89'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:37.438682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f531d0fe3d64ff6ba4d2e960f5ee84a', 'request_id': 'req-1e9924f6-57cb-4005-84f0-1c121f0a0d89'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:37.474682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] The token cache is being invalidate because project 9f531d0fe3d64ff6ba4d2e960f5ee84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:37.476492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:37.478303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1e9924f6-57cb-4005-84f0-1c121f0a0d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:37.485863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 819/1655] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:37 2022] DELETE /identity/v3/projects/9f531d0fe3d64ff6ba4d2e960f5ee84a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:37.487482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.487778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.488061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.516393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:37.520079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:37.657734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.658462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.665599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.665738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 837/1656] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:37.683430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:37.685078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:37.699989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:37.711020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 820/1657] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:37.722271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3204-135d-4eaa-9718-a89e16cbd503 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:37.729481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:37.730764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.731097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.734478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.734478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:37.734478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:37.734478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:37.766606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 838/1658] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:43:37.767985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:37.782893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750e14d-57a4-4d60-95b6-4d061040dd39 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:37.814554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:37.815369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:37.817475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:37.817475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:37.817475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:37.821967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:37.967795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:37.978849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dba58b-21d6-4315-8548-b5c255d8f63a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:37.979100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 821/1659] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:37 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:38.019653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:38.064492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:38.065614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.066119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.066625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.067570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:38.070144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:38.221298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 839/1660] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:37 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:38.223119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:38.240058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3a15-0d9a-4979-93d2-1b8c4136bcac admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:38.252659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:38.253355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.253643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.254529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.254664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:38.256364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:38.284266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 822/1661] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:38.284361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:38.292471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc34b9-52e2-4337-901c-61a2eb63097c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:38.300084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:38.301103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.301796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.302286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.304185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:38.307031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593b982-90f6-49c3-b072-876630dd769b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:38.329343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 840/1662] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:38.333580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:38.356413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319b89f-9d88-4c7c-9383-77d3e8069278 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:38.370201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:38.371671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.371960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.372244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] PATH_INFO: `/v3/projects/a5742f8a1bc94594ae52ac814bc306cf/users/8b1e43e5d56f4085af872e7c794dcb9b/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.414613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5742f8a1bc94594ae52ac814bc306cf, user_id=8b1e43e5d56f4085af872e7c794dcb9b, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:38.417112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:38.475573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:38.505118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 823/1663] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:38 2022] PUT /identity/v3/projects/a5742f8a1bc94594ae52ac814bc306cf/users/8b1e43e5d56f4085af872e7c794dcb9b/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:38.520290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.520677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.520924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.563285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] MFA Rules not processed for user `8b1e43e5d56f4085af872e7c794dcb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:38.567127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:38.726370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 841/1664] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:38.731516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:38.908979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0221-19cd-4726-b975-bf32d41b19c8 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:38.923241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:38.924786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:38.925736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:38.926855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:38.927941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:38.930892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:39.072875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 824/1665] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:38 2022] POST /identity/v3/projects => generated 373 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:39.077113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:39.090584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91397fd-d777-4ef1-8268-1201c0781877 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:39.221509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:39.222615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:39.223322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:39.225570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:39.228323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:39.263051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:39.387782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 842/1666] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:39 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:39.394691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:39.413180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b401aa7-44e2-42f8-a27e-56d7c102f2c7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:39.431772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:39.434049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:39.439621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:39.440524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:39.440955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:39.441383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:39.442268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:39.450207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:39.463472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8fab6-c0b2-4350-ab63-793351d0c53d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:39.491705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:39.492619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:39.496457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:39.496799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] PATH_INFO: `/v3/users/eb090bd3ce3741268abd614d943d21fb` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:39.502285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb090bd3ce3741268abd614d943d21fb)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:39.503374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: WARNING py.warnings [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 19:43:39.503374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: warnings.warn(msg) Jun 16 19:43:39.503374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: Jun 16 19:43:39.505111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:39.520364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 825/1667] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:39.525636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:39.549076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e484-3c27-4478-af65-264a2441d8ff admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:39.567939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:39.568855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:39.570647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:39.571171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] PATH_INFO: `/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4/users/89ed13b30b714e0ebfaef87f65aec581/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:39.609048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0971b1cc698f44d4b8ee15fdf4f38df4, user_id=89ed13b30b714e0ebfaef87f65aec581, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:39.614173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:39.769495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 826/1668] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:39 2022] PUT /identity/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4/users/89ed13b30b714e0ebfaef87f65aec581/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:39.773926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:39.843209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:39.887534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb090bd3ce3741268abd614d943d21fb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:39.888287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb090bd3ce3741268abd614d943d21fb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:39.906808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb090bd3ce3741268abd614d943d21fb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:39.969296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbc6eb-6848-4e4e-8841-ca25962999a4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:39.985465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:39.986371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:39.986835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:39.987396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:39.988347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:39.990160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:40.016777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb090bd3ce3741268abd614d943d21fb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:40.019196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 827/1669] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:40.023316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:40.025338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:40.060208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7d9dd66-0b69-45b0-b41c-045507e2a8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb090bd3ce3741268abd614d943d21fb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:40.064437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726b0b-c6af-4812-a6e4-b8a206b520d3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:40.079742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:40.080760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:40.081156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:40.081520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] PATH_INFO: `/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4/users/89ed13b30b714e0ebfaef87f65aec581/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:40.082026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 843/1670] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:39 2022] DELETE /identity/v3/users/eb090bd3ce3741268abd614d943d21fb => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:40.086633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:40.086853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0971b1cc698f44d4b8ee15fdf4f38df4, user_id=89ed13b30b714e0ebfaef87f65aec581, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:40.099147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:40.099147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:40.099683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6a3e7-c8ca-4f5c-b3d3-c7d75e5eb64f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:40.119770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 828/1671] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:40 2022] PUT /identity/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4/users/89ed13b30b714e0ebfaef87f65aec581/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:40.139308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:40.140569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:40.145476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:40.147709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:40.148954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:40.205263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] MFA Rules not processed for user `89ed13b30b714e0ebfaef87f65aec581`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:40.206688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:40.257700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:40.336499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 844/1672] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:43:40.360748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:40.362413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:40.418902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 829/1673] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:40.431307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:40.436148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:40.468770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52bccd-a505-4f15-8d09-f20cfc24d9a9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:40.480864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:40.481557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:40.481850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:40.482128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:40.510884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:40.512383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:40.528385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 845/1674] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:43:40.601696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1e8c-8937-48db-86b1-2b64bee545ed None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:40.611208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:40.611885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:40.612188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:40.612489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:40.639128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:40.641408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:40.676705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 830/1675] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:40.975176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:40.992113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fbe7-bf87-41ad-8978-120b458ebcb0 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:41.011822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:41.014165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:41.014165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:41.014165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:41.026322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:41.220626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:41.227203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e34ff-8a38-4c69-a192-f516078eceaf service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:41.241379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 846/1676] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:41.415841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:41.940382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:41.942537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:41.968906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d834aa-4331-4303-bfdd-7fbbb0540ab8 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:41.974563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:41.975642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:41.976093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:41.976511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] PATH_INFO: `/v3/projects/19487af2cbd5445d88ca0df1e308ccc3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:41.978487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19487af2cbd5445d88ca0df1e308ccc3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:41.980848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:42.236831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19487af2cbd5445d88ca0df1e308ccc3', 'request_id': 'req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:42.262439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19487af2cbd5445d88ca0df1e308ccc3', 'request_id': 'req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:42.369901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19487af2cbd5445d88ca0df1e308ccc3', 'request_id': 'req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:42.389130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] The token cache is being invalidate because project 19487af2cbd5445d88ca0df1e308ccc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:42.389435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:42.390844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e35670b9-30d5-417a-9fc0-fd7f8a9a3627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:42.395007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 831/1677] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:41 2022] DELETE /identity/v3/projects/19487af2cbd5445d88ca0df1e308ccc3 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:42.400793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2172-836b-4512-938c-f7f190561957 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:42.401285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2172-836b-4512-938c-f7f190561957 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:42.402674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2172-836b-4512-938c-f7f190561957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:42.448321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a5dd2172-836b-4512-938c-f7f190561957 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:42.450118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a5dd2172-836b-4512-938c-f7f190561957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:42.647231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 847/1678] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:42.653838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:42.654139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:42.659351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:42.692426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:42.692597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:42.694282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:42.702673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:42.702673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:42.714906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 832/1679] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:42.721021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:42.739908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497eb6-292b-4c69-8d2b-9e0dbf83850e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:42.752174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:42.753110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:42.753547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:42.753974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:42.754955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:42.755436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:42.757222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:42.763603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] MFA Rules not processed for user `8b5a188cee0d4727a14153088b8d95c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:42.795126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:42.804106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 833/1680] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:43:42.811372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:42.841639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec3c8-0c0c-4f1d-b309-35e661064a0f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:42.864569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:42.865251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:42.865566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:42.868908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:42.868908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:42.868908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:42.979593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 848/1681] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:42.987902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:42.991573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:42.997995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 834/1682] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:42 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:43.002106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:43.008814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:43.161622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62efce-6b0f-4932-9af0-4186bc6a10bd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:43.168380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b3e5-4bec-4ec0-8ad9-bcfe4c521bb2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:43.176008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:43.176764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.177949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.178555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.185942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:43.191113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:43.194162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:43.266552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:43.280691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.281069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.281362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.352907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 835/1683] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:43 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:43.354077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:43.376460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c699f7-009c-480a-83f1-16b0893b07bd admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:43.376968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:43.380538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:43.396512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:43.397684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.398156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.398579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.399824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:43.401933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:43.406936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 849/1684] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:43.424634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:43.434325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 836/1685] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:43.438326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7277e-2d9f-400a-a2fe-ed32722f20d6 service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:43.445632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:43.446576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.447212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.447732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.449087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:43.451232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1da82d-d12e-4154-813c-68e85f22cdc6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:43.480443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:43.488886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 850/1686] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:43.494480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fca96-3d62-4660-8327-810de60c5228 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:43.504911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:43.506211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.506731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.507768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] PATH_INFO: `/v3/projects/dee51b9fc703490b80a54ec370d7d726/users/a30204abeec8403d88e922f46636cc0f/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.530727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee51b9fc703490b80a54ec370d7d726, user_id=a30204abeec8403d88e922f46636cc0f, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:43.534622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:43.632216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 837/1687] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:43 2022] PUT /identity/v3/projects/dee51b9fc703490b80a54ec370d7d726/users/a30204abeec8403d88e922f46636cc0f/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:43.633262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:43.633762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:43.634222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:43.698768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] MFA Rules not processed for user `a30204abeec8403d88e922f46636cc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:43.700215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:43.845077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 851/1688] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:43.849559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.102753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82425de1-7c1f-4642-8734-f5c43aa20d9e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:44.112876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:44.114021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:44.114508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:44.115674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:44.116983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:44.119807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:44.241272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 838/1689] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:43 2022] POST /identity/v3/projects => generated 373 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:44.242536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.257584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e250d36-7804-47b4-8717-90c1792d2a10 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:44.371214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:44.443733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:44.445591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:44.446010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:44.446429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:44.447327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:44.451581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:44.603621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 852/1690] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:44 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:43:44.608210 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.622958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d52a-c183-458a-ab6a-946577492e18 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:44.638738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:44.640042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:44.640042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:44.640042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:44.640735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:44.642536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:44.690878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.703477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 839/1691] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:44.709834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ec-0527-49d0-bdc3-2d4772d7749a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:44.731610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:44.732663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:44.733385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:44.733385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] PATH_INFO: `/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:44.768723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ee44daf8d446aa9d49495eb3c2e3d4, user_id=ff5f91c9e7314b4a94cfc0e62c2f6ca6, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:44.771895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:44.873615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 853/1692] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:44 2022] PUT /identity/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:44.874770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.890260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1a87-79bb-4bc9-b86a-c00b1cccc9ae admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:44.984958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:44.988884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:45.015339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:45.105749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:45.106599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:45.106890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:45.109270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:45.109270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:45.123565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:45.155677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 840/1693] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:45.160101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:45.206579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47e43-0172-4394-968b-effdc66d221c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:45.216756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:45.217491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:45.217776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:45.218057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:45.226708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:45.260493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ac73-962c-47c6-aac6-ab211a9cd448 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:45.290425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:45.291502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:45.291699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:45.295520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] PATH_INFO: `/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:45.299375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ee44daf8d446aa9d49495eb3c2e3d4, user_id=ff5f91c9e7314b4a94cfc0e62c2f6ca6, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:45.336985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0e68c-604b-4dfb-805a-dabff54ed84c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:45.398277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 841/1694] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:45 2022] PUT /identity/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:45.420277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7d8-4780-4fe0-8907-8b7e0ab27e58 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:45.420524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7d8-4780-4fe0-8907-8b7e0ab27e58 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:45.420768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7d8-4780-4fe0-8907-8b7e0ab27e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:45.548007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:45.551741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:45.559523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-75adf7d8-4780-4fe0-8907-8b7e0ab27e58 None None] MFA Rules not processed for user `ff5f91c9e7314b4a94cfc0e62c2f6ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:45.560980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-75adf7d8-4780-4fe0-8907-8b7e0ab27e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:45.579599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 854/1695] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:43:45.855522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 842/1696] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:46.064561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:46.066432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:46.084899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca380-16c3-42c1-8131-9f8ffd242710 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:46.167365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88020c-80b1-417e-a6d8-ff52524f2517 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:46.168794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88020c-80b1-417e-a6d8-ff52524f2517 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:46.168794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88020c-80b1-417e-a6d8-ff52524f2517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:46.259775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1f88020c-80b1-417e-a6d8-ff52524f2517 None None] MFA Rules not processed for user `4df05cbf303c4524a665b429f240a17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:46.262326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1f88020c-80b1-417e-a6d8-ff52524f2517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:46.285375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:46.285804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:46.286133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:46.286339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:46.316163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:46.323705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:46.325146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:46.363625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 855/1697] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:46.434449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 843/1698] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:46.449661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:46.451141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:46.620169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8ceff-9bd6-4bb6-a92c-156a8564f94d service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:46.635103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:46.635842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:46.636129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:46.636408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:46.654097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:46.660683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b427da3-7981-439f-a27e-cca21b67ec1a service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:46.681647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 856/1699] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:48.214706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:48.215898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:48.229121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:48.285270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] MFA Rules not processed for user `d9320ca5b2a9499a86d5614b55eddede`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:48.290824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:48.375616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:48.570021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 844/1700] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:48.582591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955baf2-d0b1-4b31-a948-d980ff075632 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:48.583243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955baf2-d0b1-4b31-a948-d980ff075632 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:48.583243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955baf2-d0b1-4b31-a948-d980ff075632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:48.634999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e955baf2-d0b1-4b31-a948-d980ff075632 None None] MFA Rules not processed for user `d9320ca5b2a9499a86d5614b55eddede`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:48.639238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e955baf2-d0b1-4b31-a948-d980ff075632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:48.676375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 857/1701] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:48.689745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:48.724158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa002258-a472-4588-a75a-9314e5de64dd None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:48.742313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:48.744528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:48.747072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:48.747568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:48.775674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:48.784555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf83ae-ad38-46c7-a8c5-f9d85fb320d0 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:48.802911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 845/1702] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:49.446578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f4a55-7905-4ea5-9c0c-a124bf940bdb None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:49.446956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f4a55-7905-4ea5-9c0c-a124bf940bdb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:49.447438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f4a55-7905-4ea5-9c0c-a124bf940bdb None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:49.449900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 858/1703] 10.222.0.36 () {54 vars in 912 bytes} [Thu Jun 16 19:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:49.457028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:49.457986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:49.459051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:49.592388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:49.605715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:49.735939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f4a55-7905-4ea5-9c0c-a124bf940bdb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:49.858046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 846/1704] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:43:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:49.922217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:49.927032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:49.964557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f4a55-7905-4ea5-9c0c-a124bf940bdb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:49.973813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168a118-2ddc-4c11-8034-cbf148208735 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:49.985815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:49.989054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:49.992965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:49.996706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:49.998243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:50.000380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:50.000896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:50.001604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] PATH_INFO: `/v3/users/a30204abeec8403d88e922f46636cc0f` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:50.004469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30204abeec8403d88e922f46636cc0f)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:50.011519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:50.019346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:50.021090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440995c2-46aa-4f0f-9fa8-411508a5dec3 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:50.061789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 847/1705] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:43:50.303163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30204abeec8403d88e922f46636cc0f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:50.303847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30204abeec8403d88e922f46636cc0f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:50.339337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30204abeec8403d88e922f46636cc0f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:50.447158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30204abeec8403d88e922f46636cc0f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:50.481842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:50.487875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30204abeec8403d88e922f46636cc0f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:50.506016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 859/1706] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:49 2022] DELETE /identity/v3/users/a30204abeec8403d88e922f46636cc0f => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:50.508058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa557-9300-4df8-aa09-bc1a5f5f6081 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:50.508547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa557-9300-4df8-aa09-bc1a5f5f6081 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:50.508759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa557-9300-4df8-aa09-bc1a5f5f6081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:50.538881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ae7aa557-9300-4df8-aa09-bc1a5f5f6081 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:50.545741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ae7aa557-9300-4df8-aa09-bc1a5f5f6081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:50.672845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 848/1707] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:50.696082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:50.699151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:50.764886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:50.765175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:50.765452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:50.830673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] MFA Rules not processed for user `2d0d5723417d4b60bc51e47f28d09902`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:50.840379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2fafe-40f0-42cc-bf2f-5b5c28f847a0 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:50.843155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:50.862258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:50.863152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:50.863607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:50.864031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:50.874818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:50.876580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f688aa-1acd-4ebb-b09d-53ea3ebf6b73 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:50.889493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 860/1708] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:50.890536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:51.010598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 849/1709] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:51.024915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc99b-6c0d-4d94-9528-493568e26493 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.025318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc99b-6c0d-4d94-9528-493568e26493 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.025876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc99b-6c0d-4d94-9528-493568e26493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.071185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8ccfc99b-6c0d-4d94-9528-493568e26493 None None] MFA Rules not processed for user `2d0d5723417d4b60bc51e47f28d09902`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:51.072902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8ccfc99b-6c0d-4d94-9528-493568e26493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:51.098036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 861/1710] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:51.107875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:51.109271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:51.299221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2813f-2b82-451e-86b1-cfb8d07cf77a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:51.311539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:51.326203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.326508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.326798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.364882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:51.368846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe3760-a95a-401c-ad42-425748567f93 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:51.424650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.424650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.424650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.438324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 850/1711] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:51.509138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f2e1-6fc6-4e91-b15b-58a8fd4c351e None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.509752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f2e1-6fc6-4e91-b15b-58a8fd4c351e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.509752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9f2e1-6fc6-4e91-b15b-58a8fd4c351e None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.512543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 851/1712] 10.222.0.36 () {54 vars in 960 bytes} [Thu Jun 16 19:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:51.522081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0bb6-9baa-44ae-be17-998e99dc7205 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.522401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0bb6-9baa-44ae-be17-998e99dc7205 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.523080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0bb6-9baa-44ae-be17-998e99dc7205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.541328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] MFA Rules not processed for user `8b5a188cee0d4727a14153088b8d95c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:51.543228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:51.582350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-70fb0bb6-9baa-44ae-be17-998e99dc7205 None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:51.582350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-70fb0bb6-9baa-44ae-be17-998e99dc7205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:51.772573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 862/1713] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:51.790117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 852/1714] 10.222.0.36 () {58 vars in 1057 bytes} [Thu Jun 16 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:43:51.800852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:51.847521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490091b-6f99-4ce0-a5bb-c87b22f9a54c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:51.862553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:51.864349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:51.864796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:51.866007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:51.873558 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:51.942176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:51.943838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1fbc3-1732-40df-825d-2eaea363533d service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:51.968844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 863/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:52.776696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:52.777358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:52.777358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:52.784050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 853/1716] 10.222.0.36 () {54 vars in 912 bytes} [Thu Jun 16 19:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:43:52.788165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:52.789345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:52.789839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:52.851079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:52.851079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:52.881275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:52.884725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3238ae-5fc3-479c-a6fb-764e39517f1b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:52.889080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:52.900715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:52.901618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:52.901888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:52.903642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] PATH_INFO: `/v3/projects/dee51b9fc703490b80a54ec370d7d726` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:52.905090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee51b9fc703490b80a54ec370d7d726)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:52.907425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:52.959823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 864/1717] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:43:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:52.967546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:52.976987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b7ff7-2d4e-444d-981a-be4289ba0595 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:52.984308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:52.985078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:52.985440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:52.985794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:52.994342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:52.996857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:53.013638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 865/1718] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:43:53.131014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:53.169168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee51b9fc703490b80a54ec370d7d726', 'request_id': 'req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.195980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee51b9fc703490b80a54ec370d7d726', 'request_id': 'req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.302217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee51b9fc703490b80a54ec370d7d726', 'request_id': 'req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.322430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] The token cache is being invalidate because project dee51b9fc703490b80a54ec370d7d726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:53.322734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.325602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8fd08cac-d66a-4bc3-b095-8855fa7f7a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.329795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 854/1719] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:52 2022] DELETE /identity/v3/projects/dee51b9fc703490b80a54ec370d7d726 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:53.333960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:53.354308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988186-d79a-48fd-9668-0b294567454a service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:53.491191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:53.492503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:53.493102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:53.493578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] PATH_INFO: `/v3/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:53.499221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5f91c9e7314b4a94cfc0e62c2f6ca6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:53.503347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:53.522326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:53.654852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:53.655237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:53.655505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:53.686586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] MFA Rules not processed for user `4df05cbf303c4524a665b429f240a17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:53.687482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:53.792747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 855/1720] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:53.873756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5f91c9e7314b4a94cfc0e62c2f6ca6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.874849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5f91c9e7314b4a94cfc0e62c2f6ca6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:53.895722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5f91c9e7314b4a94cfc0e62c2f6ca6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:54.003750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5f91c9e7314b4a94cfc0e62c2f6ca6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:54.086550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5f91c9e7314b4a94cfc0e62c2f6ca6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:54.117509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 866/1721] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:53 2022] DELETE /identity/v3/users/ff5f91c9e7314b4a94cfc0e62c2f6ca6 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:54.396118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:54.398356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:54.433673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b91c8-2b72-4c12-9a41-4f3dda901cd4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:54.553597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:54.559853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:54.630568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:54.631609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:54.635014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:54.635291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] PATH_INFO: `/v3/users/d1dca219a643470590ab0325e705e037` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:54.636681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dca219a643470590ab0325e705e037)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:54.639984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:54.734856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca02840-179d-4e90-89ad-beb9e613f1a3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:54.742803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:54.743742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:54.744086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:54.744376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] PATH_INFO: `/v3/users/2183fd35fd7c412cb5bbdfa24a0c91db` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:54.748340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2183fd35fd7c412cb5bbdfa24a0c91db)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:54.753155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:54.948703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dca219a643470590ab0325e705e037'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:54.957823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dca219a643470590ab0325e705e037'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:54.977017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dca219a643470590ab0325e705e037'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.072239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dca219a643470590ab0325e705e037'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.078325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:55.078916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2183fd35fd7c412cb5bbdfa24a0c91db'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.083432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2183fd35fd7c412cb5bbdfa24a0c91db'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.091879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16763ccd-2338-427e-b943-e18e701f0781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dca219a643470590ab0325e705e037'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.109535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2183fd35fd7c412cb5bbdfa24a0c91db'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.123028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 856/1722] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:54 2022] DELETE /identity/v3/users/d1dca219a643470590ab0325e705e037 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:55.132777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85327b8-5586-4bcb-9c04-5a2bd2729a40 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.133274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85327b8-5586-4bcb-9c04-5a2bd2729a40 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.135003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85327b8-5586-4bcb-9c04-5a2bd2729a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.176279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f85327b8-5586-4bcb-9c04-5a2bd2729a40 None None] MFA Rules not processed for user `2d0d5723417d4b60bc51e47f28d09902`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:55.177918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f85327b8-5586-4bcb-9c04-5a2bd2729a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.228612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2183fd35fd7c412cb5bbdfa24a0c91db'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.259533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-103c623b-7ac4-45de-aa8b-afeec685c98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2183fd35fd7c412cb5bbdfa24a0c91db'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.270641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 857/1723] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:55.271936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:55.273000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.273487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.273935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.288156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 867/1724] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:54 2022] DELETE /identity/v3/users/2183fd35fd7c412cb5bbdfa24a0c91db => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:43:55.296019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.302505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.302802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.304293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:55.306108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.376120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:55.386127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.391253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 858/1725] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:55.399412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:55.404705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.419248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 868/1726] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:55.437885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:55.437885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.541726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1660-2a67-4690-9c45-6bccb9b5aeac None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:55.546047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:55.546775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.547567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.547982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.550197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b54410-a628-45c5-8e84-a3863a5092a1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:55.562432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:55.563779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:55.565271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:55.567228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.567671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.568064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.596905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:55.598450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768de070-48cf-4d37-b059-b18f6f8b42b2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:55.601364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 869/1727] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:55.607525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:55.612351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:55.631742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 859/1728] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:43:55.647950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eaebb-185b-4cba-87fb-687fae10a186 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:55.668470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:55.669364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:55.669772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:55.670073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] PATH_INFO: `/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:55.673103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ee44daf8d446aa9d49495eb3c2e3d4)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:55.676446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:55.947790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ee44daf8d446aa9d49495eb3c2e3d4', 'request_id': 'req-fac33653-8073-4bcb-bc33-39b3c8944ba1'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:55.969948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ee44daf8d446aa9d49495eb3c2e3d4', 'request_id': 'req-fac33653-8073-4bcb-bc33-39b3c8944ba1'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:56.085652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ee44daf8d446aa9d49495eb3c2e3d4', 'request_id': 'req-fac33653-8073-4bcb-bc33-39b3c8944ba1'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:56.116433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] The token cache is being invalidate because project 73ee44daf8d446aa9d49495eb3c2e3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:56.116737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:56.118657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-fac33653-8073-4bcb-bc33-39b3c8944ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:56.124372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 870/1729] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:55 2022] DELETE /identity/v3/projects/73ee44daf8d446aa9d49495eb3c2e3d4 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:56.126186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:56.126674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:56.126835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:56.200148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:56.205578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:56.369456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 860/1730] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:56.378845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:56.379295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:56.379677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:56.411563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:56.413575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:56.428874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 871/1731] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:43:56.429393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:56.441612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481041c1-a542-41a6-84ca-08d0282aae00 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:56.455865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:56.456511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:56.456786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:56.457072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:56.457831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:43:56.458145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:56.461907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:56.489593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 861/1732] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:43:56.495895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:56.519166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ecd6c-efe7-4544-a6ab-78e084886155 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:56.535361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:56.536063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:56.536358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:56.537466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:56.537466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:56.542677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:56.733353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 872/1733] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:56 2022] POST /identity/v3/projects => generated 355 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:56.737214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:56.754385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e631-9c23-4baf-a563-3a7c32eab720 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:56.923281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:56.925663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:57.011782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:57.012488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:57.013117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:57.013410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:57.015456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:57.022389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:57.164990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 862/1734] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:56 2022] POST /identity/v3/users => generated 278 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:57.166868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:57.190170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afd8c4-8720-48d8-aa03-b0195675d7c3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:57.214726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:57.217782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:57.218356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:57.219033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:57.219958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:57.222805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:57.241866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351ae8c-1727-4a91-bfc1-daf3deacac83 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:57.262804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 863/1735] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:57.264689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:57.276536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:57.279354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:57.283610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:57.287888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:57.288405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] PATH_INFO: `/v3/projects/83a154e7e1db49adb3d342788331e0e5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:57.293292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a154e7e1db49adb3d342788331e0e5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:57.314203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:57.328941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2a24-18ac-4d76-b2ae-83d250a121a0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:57.356747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:57.360907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:57.366441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:57.367279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] PATH_INFO: `/v3/projects/df6beb24a9cc41dba4182b369e045f1a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:57.377793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6beb24a9cc41dba4182b369e045f1a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:57.380117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:57.540183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a154e7e1db49adb3d342788331e0e5', 'request_id': 'req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.555909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a154e7e1db49adb3d342788331e0e5', 'request_id': 'req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.684769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a154e7e1db49adb3d342788331e0e5', 'request_id': 'req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.739772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] The token cache is being invalidate because project 83a154e7e1db49adb3d342788331e0e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:57.742970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.745969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.749290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 873/1736] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:56 2022] DELETE /identity/v3/projects/83a154e7e1db49adb3d342788331e0e5 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:57.754476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:57.757363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:57.763123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6beb24a9cc41dba4182b369e045f1a', 'request_id': 'req-df831ebd-ec03-4ec5-b667-46087a4a2c72'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.786661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6beb24a9cc41dba4182b369e045f1a', 'request_id': 'req-df831ebd-ec03-4ec5-b667-46087a4a2c72'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.881250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd012fd-1557-4d73-a8c5-b54b39f2d12a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:57.926142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6beb24a9cc41dba4182b369e045f1a', 'request_id': 'req-df831ebd-ec03-4ec5-b667-46087a4a2c72'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.975885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:57.977020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:57.977409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:57.978191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:57.979219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:57.981553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:57.993960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] The token cache is being invalidate because project df6beb24a9cc41dba4182b369e045f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:43:57.994260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.998703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 874/1737] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:57.999002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:57.999993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:58.002805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 864/1738] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:43:57 2022] DELETE /identity/v3/projects/df6beb24a9cc41dba4182b369e045f1a => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:58.004394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.005148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:58.009929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.023628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f423096-67a9-4f61-a4ab-6b27fa966dea admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:58.031200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:58.033510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:58.033510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:58.039543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] PATH_INFO: `/v3/users/5223ce69f5604973b55299f794375971` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:58.042507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5223ce69f5604973b55299f794375971)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:58.045244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:58.048474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831ebd-ec03-4ec5-b667-46087a4a2c72 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:58.064638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:58.065417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:58.080182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:58.080182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] PATH_INFO: `/v3/projects/5fe350abc5364ceab16d16f9e78d35ba/users/2267e41c80c84e9eb463fab2d08723c2/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:58.128032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe350abc5364ceab16d16f9e78d35ba, user_id=2267e41c80c84e9eb463fab2d08723c2, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:58.132897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:58.306135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 865/1739] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:58 2022] PUT /identity/v3/projects/5fe350abc5364ceab16d16f9e78d35ba/users/2267e41c80c84e9eb463fab2d08723c2/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:58.308319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:58.310351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.364227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5223ce69f5604973b55299f794375971'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.365904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5223ce69f5604973b55299f794375971'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.377035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5223ce69f5604973b55299f794375971'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.460493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa69-18ce-43a1-80b3-009d5b8b9631 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:58.464567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:58.476042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:58.477744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:58.478561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:58.479141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] PATH_INFO: `/v3/users/e08d385b94cd44f1af9a54f349d74731` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:58.483349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08d385b94cd44f1af9a54f349d74731)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:58.487482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:58.499195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5223ce69f5604973b55299f794375971'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.533161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ed08ed11-28d7-45fa-af9a-828ecfb465aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5223ce69f5604973b55299f794375971'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.583951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 875/1740] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:57 2022] DELETE /identity/v3/users/5223ce69f5604973b55299f794375971 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:58.583951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37e65f-9270-449e-8724-6f4dbce0a4b0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:58.583951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37e65f-9270-449e-8724-6f4dbce0a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:58.583951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37e65f-9270-449e-8724-6f4dbce0a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:58.618084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:43:58.640589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fc37e65f-9270-449e-8724-6f4dbce0a4b0 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:58.641866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fc37e65f-9270-449e-8724-6f4dbce0a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.729268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 876/1741] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:58.730610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:58.730910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:58.731218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:58.768882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] MFA Rules not processed for user `2267e41c80c84e9eb463fab2d08723c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:58.771200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.886771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 877/1742] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:58.889368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:58.890821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:58.930009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08d385b94cd44f1af9a54f349d74731'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.931060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08d385b94cd44f1af9a54f349d74731'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.948136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08d385b94cd44f1af9a54f349d74731'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:58.997261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbf8c3-e7e6-4180-99e9-2c6d0969257e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.012263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.013052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.013640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.014019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.014793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:59.020363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:59.077757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08d385b94cd44f1af9a54f349d74731'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:59.102878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08d385b94cd44f1af9a54f349d74731'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:43:59.110356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 866/1743] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:43:58 2022] DELETE /identity/v3/users/e08d385b94cd44f1af9a54f349d74731 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:43:59.208474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:59.209895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 878/1744] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:43:58 2022] POST /identity/v3/projects => generated 355 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:59.214016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:59.214481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:59.217596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:59.412438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb0c1-9503-4ddc-810c-8fedb86f0a1f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.430175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.431146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.431579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.433302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.435239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:59.437098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b65b1-6e62-4c54-880a-3f833adf97bb admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.440593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ff8b6-27b2-4abc-8707-dd5e18524a5b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:59.540871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.541607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.541993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.542385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.544857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:59.616837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 879/1745] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:43:59 2022] POST /identity/v3/users => generated 284 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:43:59.626587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.626968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.627665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.677609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:59.679698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:59.682854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:43:59.684156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:59.714289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 867/1746] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:43:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:43:59.740145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:59.792947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc36a8-7dd0-4a3c-8c93-d487322994ed service swift] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.803419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.804224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.805002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.805002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.805667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:59.809287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:59.826822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 868/1747] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:43:59.831205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:59.845902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4562d-6026-4bc0-b26e-41b495da2dc6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.853361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.854729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.855153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.855502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] PATH_INFO: `/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877/users/2310d9c6393b4b9d806547ddb9341396/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:43:59.875966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 880/1748] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Jun 16 19:43:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:43:59.878179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbd2d63f4ec44388bf73c9ea0c47877, user_id=2310d9c6393b4b9d806547ddb9341396, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:43:59.883262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:43:59.948144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:43:59.949617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:43:59.960259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95284dc-6bf2-41fa-be6a-dec3806940b4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:43:59.972062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:43:59.972993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:43:59.973433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:43:59.973886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.009563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:00.014381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:00.015264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 869/1749] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:43:59 2022] PUT /identity/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877/users/2310d9c6393b4b9d806547ddb9341396/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:00.027946 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:00.066677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 881/1750] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:43:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:00.070172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:00.071889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:00.126120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad240f3-ea0a-4ada-9d49-e8b7cd28cc79 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:00.166138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd701feb-fa1c-4817-826e-81024287aeb6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:00.187948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:00.187948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:00.187948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:00.187948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.188572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:00.198993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:00.258452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 870/1751] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:00.273807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:00.300244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ccbe1-35dc-4773-83fe-9e0961693726 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:00.317066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:00.318089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:00.318374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:00.318643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] PATH_INFO: `/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877/users/2310d9c6393b4b9d806547ddb9341396/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.319316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:00.321277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbd2d63f4ec44388bf73c9ea0c47877, user_id=2310d9c6393b4b9d806547ddb9341396, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:00.324615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbf683-3704-4888-8279-add766f18c70 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:00.324976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:00.325494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:00.326032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] PATH_INFO: `/v3/projects/9bd1bc122ce843c1b54a632993f24bc7` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.327684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd1bc122ce843c1b54a632993f24bc7)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:00.331206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:00.352707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 871/1752] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:00 2022] PUT /identity/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877/users/2310d9c6393b4b9d806547ddb9341396/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:00.368449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c5425-deff-453c-a36b-0e027de29541 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:00.368917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c5425-deff-453c-a36b-0e027de29541 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:00.375410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c5425-deff-453c-a36b-0e027de29541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.484517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-080c5425-deff-453c-a36b-0e027de29541 None None] MFA Rules not processed for user `2310d9c6393b4b9d806547ddb9341396`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:00.486842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-080c5425-deff-453c-a36b-0e027de29541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:00.603210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd1bc122ce843c1b54a632993f24bc7', 'request_id': 'req-8d603b69-970d-4334-ade7-3d3aa0bc01f6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:00.611407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:00.615130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd1bc122ce843c1b54a632993f24bc7', 'request_id': 'req-8d603b69-970d-4334-ade7-3d3aa0bc01f6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:00.741535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 872/1753] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:00 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:00.741535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c5425-deff-453c-a36b-0e027de29541 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:00.748323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-080c5425-deff-453c-a36b-0e027de29541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:00.763807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd1bc122ce843c1b54a632993f24bc7', 'request_id': 'req-8d603b69-970d-4334-ade7-3d3aa0bc01f6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:00.787462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] The token cache is being invalidate because project 9bd1bc122ce843c1b54a632993f24bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:00.788149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:00.789388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:00.792963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 882/1754] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:00 2022] DELETE /identity/v3/projects/9bd1bc122ce843c1b54a632993f24bc7 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:00.795710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:00.798030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:00.917198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603b69-970d-4334-ade7-3d3aa0bc01f6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:00.923068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:00.923993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:00.925124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:00.925124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:00.927214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:00.972480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c5425-deff-453c-a36b-0e027de29541 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb0830-684c-469c-998a-a9549be3eced service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 883/1755] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:00 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0cfc7-9552-4471-9b8f-75279895e488 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0cfc7-9552-4471-9b8f-75279895e488 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:01.344860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0cfc7-9552-4471-9b8f-75279895e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:01.348372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:01.348372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:01.348372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:01.348372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] PATH_INFO: `/v3/projects/31b8a8b5ec064e2880d21b147ea04589` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:01.349842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b8a8b5ec064e2880d21b147ea04589)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:01.349842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d9c0cfc7-9552-4471-9b8f-75279895e488 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d9c0cfc7-9552-4471-9b8f-75279895e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 884/1756] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:01.350575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:01.379770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:01.469495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:01.469495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 885/1757] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:01.482165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:01.514411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d332b9-7f4d-46e0-88dd-f300eed31d62 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:01.527216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:01.528006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:01.528300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:01.528580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:01.529345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:01.533724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:01.535407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:01.606965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 886/1758] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:01.610263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:01.640602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01183da-e401-4891-bcce-df705196e496 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:01.648211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:01.653073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:01.653356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:01.654019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:01.654741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:01.656863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:01.799414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b8a8b5ec064e2880d21b147ea04589', 'request_id': 'req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:01.829992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b8a8b5ec064e2880d21b147ea04589', 'request_id': 'req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:01.983365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:02.030535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b8a8b5ec064e2880d21b147ea04589', 'request_id': 'req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:02.036145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 887/1759] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:01 2022] POST /identity/v3/projects => generated 377 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:44:02.037685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:02.103389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] The token cache is being invalidate because project 31b8a8b5ec064e2880d21b147ea04589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:02.109380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:02.120759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:02.129662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 873/1760] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:00 2022] DELETE /identity/v3/projects/31b8a8b5ec064e2880d21b147ea04589 => generated 0 bytes in 1391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:02.141605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:02.144380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:02.282359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b554887-1009-4b7a-a7ec-682ee33fa8af admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:02.299775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:02.300711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:02.301328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:02.301557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:02.303849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:02.561160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:02.561160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:02.563170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ce8a-eb6c-48dc-bcb7-e4e7c6410af5 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:02.567719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 888/1761] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:02 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:02.568891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:02.574165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:02.764611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab9038-4c5b-437e-a353-59296ab1766f service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:02.770424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:02.771410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:02.772155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:02.772579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:02.774227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:02.784082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:02.784528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:02.786266 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:02.786777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:02.787353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] PATH_INFO: `/v3/users/93b6a5e7e2484b3ba4e8789f0dacecf9` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:02.789300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b6a5e7e2484b3ba4e8789f0dacecf9)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:02.813226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:03.024139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 874/1762] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:02 2022] POST /identity/v3/users => generated 289 bytes in 893 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:03.033575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:03.066313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07912c-0f3d-4c00-8049-6625e7b8f657 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:03.080544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:03.087232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.087232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.087232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.087232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:03.087232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:03.106169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 875/1763] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:03.124060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:03.139503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8604-0d65-4271-84cc-d19124b5c724 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:03.145390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:03.150870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.150870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.150870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.150870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:03.150870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:03.165084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 876/1764] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:03.170813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:03.185442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53be24-fd24-4339-9314-6c1b78ae0230 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:03.195690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:03.198244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.199360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.200321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] PATH_INFO: `/v3/projects/1e51131da4ff475ab3fae4fa3b2b3b42/users/c0037f95e6cd478283e1f44dd1b1b7f0/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.232306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e51131da4ff475ab3fae4fa3b2b3b42, user_id=c0037f95e6cd478283e1f44dd1b1b7f0, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:03.237152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df49ed-d791-43fc-a438-0455cd422094 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:03.292168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b6a5e7e2484b3ba4e8789f0dacecf9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:03.293505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b6a5e7e2484b3ba4e8789f0dacecf9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:03.308059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b6a5e7e2484b3ba4e8789f0dacecf9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:03.379597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 877/1765] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:03 2022] PUT /identity/v3/projects/1e51131da4ff475ab3fae4fa3b2b3b42/users/c0037f95e6cd478283e1f44dd1b1b7f0/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:03.382164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd26f-a770-464a-b58f-2640becc1d16 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.382625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd26f-a770-464a-b58f-2640becc1d16 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.383487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd26f-a770-464a-b58f-2640becc1d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.414603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-2b7bd26f-a770-464a-b58f-2640becc1d16 None None] MFA Rules not processed for user `c0037f95e6cd478283e1f44dd1b1b7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:03.415781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2b7bd26f-a770-464a-b58f-2640becc1d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:03.439213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b6a5e7e2484b3ba4e8789f0dacecf9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:03.482988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b6a5e7e2484b3ba4e8789f0dacecf9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:03.499636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 889/1766] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:02 2022] DELETE /identity/v3/users/93b6a5e7e2484b3ba4e8789f0dacecf9 => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:03.523524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 878/1767] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:03.532847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:03.534303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:03.709158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29164c-a823-4c2d-b2b5-094f63d97517 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.709158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29164c-a823-4c2d-b2b5-094f63d97517 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.709158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29164c-a823-4c2d-b2b5-094f63d97517 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.710970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 879/1768] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:44:03.711946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c500461-6488-46e6-af44-48e5e180c36a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:03.719529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:03.720015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.720316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.720647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.721506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:03.723936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:03.730095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:03.740768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:03.741043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:03.741749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:03.779552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:03.779809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:03.866474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 890/1769] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:03 2022] POST /identity/v3/projects => generated 377 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:03.871853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:03.979269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 880/1770] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:44:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:04.106068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3a127-cd47-4ecb-bf97-cca163f5ad85 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.120451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.121175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.121626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.131072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.131836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.133912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.258839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 891/1771] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:03 2022] POST /identity/v3/users => generated 295 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:04.258990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:04.290843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe690fec-207a-4177-8eb6-649e11250040 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.306869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.308374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.308664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.308962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.310452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.318327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.344862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:04.349897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:04.381396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 881/1772] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:04.383160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:04.412147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c7ed1-8519-4fb8-bfe7-465f0f038542 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.438903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.448305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.448696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.454323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] PATH_INFO: `/v3/projects/5ca390c9869741e0b58ab69552308c36/users/7b2844d455b5442e8afe38fb7bb5b21f/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.515171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca390c9869741e0b58ab69552308c36, user_id=7b2844d455b5442e8afe38fb7bb5b21f, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.523068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.628950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 882/1773] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:04 2022] PUT /identity/v3/projects/5ca390c9869741e0b58ab69552308c36/users/7b2844d455b5442e8afe38fb7bb5b21f/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:04.632169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f203b3-188f-46ec-87d5-b8b7f0fcd505 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.635200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:04.816536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21db8-4cdb-4cb7-878f-d62c306e6ff3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.841539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.842269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.842560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.842818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.843590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.845213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.887721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 883/1774] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:04.888716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:04.911876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4576b7-e71f-4531-b3dc-fc549d31c5be admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:04.934053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.934820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.935144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.935421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] PATH_INFO: `/v3/projects/5ca390c9869741e0b58ab69552308c36/users/7b2844d455b5442e8afe38fb7bb5b21f/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.937493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca390c9869741e0b58ab69552308c36, user_id=7b2844d455b5442e8afe38fb7bb5b21f, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.940253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a91458-0e67-4277-a56f-393bfc7ee4f1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.953404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:04.954710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.955171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.955597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:04.972401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 884/1775] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:04 2022] PUT /identity/v3/projects/5ca390c9869741e0b58ab69552308c36/users/7b2844d455b5442e8afe38fb7bb5b21f/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:04.980247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:04.982053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:04.984488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:04.993206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:04.993206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:05.022153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 892/1776] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:05.046325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] MFA Rules not processed for user `7b2844d455b5442e8afe38fb7bb5b21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:05.048077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:05.068756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:05.155291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 885/1777] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:05.169054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:05.170641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:05.322246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bf51f-3c96-418c-ab82-65a643c453ac service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:05.440809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:05.442027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:05.442390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:05.442830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:05.614834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:05.618051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350d717-9e1d-4234-ba5e-d11a03721c51 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:05.647047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 893/1778] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:06.501730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:06.504018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:06.663306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78c341-a772-4196-9e13-29c9fad2dedf None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:06.669884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:06.670866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:06.671693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:06.671983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] PATH_INFO: `/v3/projects/7b8120e381004f30bbc04c17af352a33` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:06.673231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b8120e381004f30bbc04c17af352a33)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:06.675568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:06.926310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b8120e381004f30bbc04c17af352a33', 'request_id': 'req-7d5714b0-1d43-446c-8240-23059e4acfbd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:06.942287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b8120e381004f30bbc04c17af352a33', 'request_id': 'req-7d5714b0-1d43-446c-8240-23059e4acfbd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:07.079851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b8120e381004f30bbc04c17af352a33', 'request_id': 'req-7d5714b0-1d43-446c-8240-23059e4acfbd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:07.106199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] The token cache is being invalidate because project 7b8120e381004f30bbc04c17af352a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:07.107402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:07.108183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7d5714b0-1d43-446c-8240-23059e4acfbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:07.113157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 886/1779] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:06 2022] DELETE /identity/v3/projects/7b8120e381004f30bbc04c17af352a33 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:07.122501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:07.124136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:07.124867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:07.149341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:07.167943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:07.335540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 894/1780] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:07.340834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:07.340834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:07.340834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:07.367401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:07.369147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:07.399139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 887/1781] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:07.404516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:07.421420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:07.426917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5649e52-bcce-4839-a909-ee63fbe45215 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:07.448625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:07.450134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:07.450772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:07.452621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:07.456225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:07.458318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:07.473089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:07.519472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 895/1782] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:07.522571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:07.535096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582cfb1-df4a-4ac4-a867-b0e954ede1a8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:07.562238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:07.563696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:07.564307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:07.564779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:07.566062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:07.575301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbcdcd-ebec-4e09-bad3-e35a54a32363 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:07.726771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 888/1783] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:07 2022] POST /identity/v3/projects => generated 391 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:07.731534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:07.762777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87928d98-00ba-4f0a-9e63-df541a017377 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:07.985300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:07.986347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:07.986563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:07.987042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.008796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:08.015468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:08.189457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319b99-5c5c-47bb-922e-f0a97aaf0bae None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.193723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319b99-5c5c-47bb-922e-f0a97aaf0bae None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.194209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319b99-5c5c-47bb-922e-f0a97aaf0bae None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.199433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 889/1784] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:44:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:44:08.243761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 896/1785] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:07 2022] POST /identity/v3/users => generated 296 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:08.247526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48319b99-5c5c-47bb-922e-f0a97aaf0bae None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:08.297211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48319b99-5c5c-47bb-922e-f0a97aaf0bae None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:08.328165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:08.328863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.329152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.329435 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.330190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:08.331857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebe098-6798-4344-8ffe-2f46be3bc81c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:08.399949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 890/1786] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:08.406061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:08.429834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0dc09-0a64-4d91-a914-28c4abca81b4 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:08.508211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:08.516223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.516223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.516223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.519398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:08.525872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:08.540934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.541581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.544003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.557715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 897/1787] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:08.562138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:08.602317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c44a1-4b13-4d0d-bd16-3a3094a28250 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:08.612845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:08.613788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.614155 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.614504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] PATH_INFO: `/v3/projects/f09ee72c9659421d9b89031b46720e23/users/80381e018d274fcd9e6350bb3a7c24b0/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.650292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:08.659769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:08.659924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09ee72c9659421d9b89031b46720e23, user_id=80381e018d274fcd9e6350bb3a7c24b0, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:08.661040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccbfa4-ea41-40dc-b76d-23d067baccdf admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:08.823007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 898/1788] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:08 2022] PUT /identity/v3/projects/f09ee72c9659421d9b89031b46720e23/users/80381e018d274fcd9e6350bb3a7c24b0/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:08.831703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:08.832243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:08.833204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:08.921001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 891/1789] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:44:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:08.925714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] MFA Rules not processed for user `80381e018d274fcd9e6350bb3a7c24b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:08.929655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:09.101024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 899/1790] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:09.107300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.124366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d65afd-9d0f-40cc-a2e6-68c187f5bc86 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:09.256428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:09.256428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:09.256428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:09.262755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:09.264700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:09.268795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:09.272918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.274393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:09.395295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336531-f593-4118-ac3c-fb618261ee3d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:09.402057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:09.403153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:09.403527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:09.403888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:09.415636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:09.417939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:09.433664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 900/1791] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:09.460308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 892/1792] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:09 2022] POST /identity/v3/projects => generated 391 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:09.464624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.475977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ca9f1-9542-4e70-9350-0787c902a1dd service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:09.491629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:09.570376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:09.571504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:09.571982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:09.572482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:09.573508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:09.575604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:09.747204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 901/1793] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:09 2022] POST /identity/v3/users => generated 302 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:09.749873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.760006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1f74-3c9b-4281-aa39-e68ca28a6a4e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:09.767363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:09.768133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:09.768498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:09.768912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:09.769882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:09.787199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:09.817708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 893/1794] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:09.818594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.832661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65296e-5716-450e-aa47-ff42378fddb5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:09.838197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:09.839524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:09.839918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:09.840308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] PATH_INFO: `/v3/projects/9e9161e45f704df4b329cea38730848f/users/916fb9390a1647a8a3a5817e49b047bc/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:09.860517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9161e45f704df4b329cea38730848f, user_id=916fb9390a1647a8a3a5817e49b047bc, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:09.863646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:09.969447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 902/1795] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:09 2022] PUT /identity/v3/projects/9e9161e45f704df4b329cea38730848f/users/916fb9390a1647a8a3a5817e49b047bc/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:09.969660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:09.978272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d39c2-a0d8-4d69-bda7-a5cfa18fcbaa admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:10.072114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:10.073439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:10.073798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:10.074176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:10.075036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:10.076739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227d2c3-140b-464c-8fb7-945fe01ea7a1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:10.090730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 894/1796] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:10.092250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:10.115460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d60e-31b0-4a5f-a70c-61d610a31a56 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:10.125854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:10.125854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:10.125854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:10.125854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] PATH_INFO: `/v3/projects/9e9161e45f704df4b329cea38730848f/users/916fb9390a1647a8a3a5817e49b047bc/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:10.127850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9161e45f704df4b329cea38730848f, user_id=916fb9390a1647a8a3a5817e49b047bc, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:10.131477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:10.151721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 903/1797] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:10 2022] PUT /identity/v3/projects/9e9161e45f704df4b329cea38730848f/users/916fb9390a1647a8a3a5817e49b047bc/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:10.161949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:10.162245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:10.162530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:10.201112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] MFA Rules not processed for user `916fb9390a1647a8a3a5817e49b047bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:10.203213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:10.353483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 895/1798] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:10.374886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:10.377647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:10.434062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:10.454573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91c3ca-1d22-486d-9031-5e9234e3af42 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:10.459494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:10.470737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:10.471223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:10.471756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:10.483958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:10.486481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:10.509062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 904/1799] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:10.628253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:10.638417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6fed9-7772-41fa-87ef-2506a2e385a2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:10.642272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:10.643146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:10.643432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:10.643711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:10.712090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:10.714004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:10.723273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 896/1800] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:12.085066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:12.896784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:13.197109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:13.199903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:13.214444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b895fb4-1d87-40e2-82c1-4c57a7cb850b service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:13.229405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:13.230154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:13.231087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:13.231379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:13.235357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:13.327088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:13.328531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de42200-0bc5-4d84-8ebc-f6ab950f8599 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:13.338627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 905/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:16.042837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:16.220009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:16.947432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:17.072416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:18.039116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:18.323005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:19.056407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:19.713211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:24.922331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:24.978559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:24.983857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f243-436f-4ced-94aa-c948b2aefe72 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:24.988365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:24.994181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:24.994181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:24.994181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] PATH_INFO: `/v3/users/80381e018d274fcd9e6350bb3a7c24b0` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:24.994181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=80381e018d274fcd9e6350bb3a7c24b0)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:24.994181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:25.204762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80381e018d274fcd9e6350bb3a7c24b0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:25.206311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80381e018d274fcd9e6350bb3a7c24b0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:25.225447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80381e018d274fcd9e6350bb3a7c24b0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:25.284814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80381e018d274fcd9e6350bb3a7c24b0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:25.300382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80381e018d274fcd9e6350bb3a7c24b0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:25.308594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 897/1802] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:24 2022] DELETE /identity/v3/users/80381e018d274fcd9e6350bb3a7c24b0 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:25.313925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:25.314391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:25.314803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:25.333611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:25.335003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:25.395698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 906/1803] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:25.405653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:25.407018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:25.479829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989467a-30c5-40b4-8423-3b8c8963e1fc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:25.483613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:25.484264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:25.484548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:25.484880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:25.494499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:25.495838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:25.505388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 898/1804] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:26.320849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:26.322328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:26.329633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:26.336134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64a9f3-dd1b-491e-8c6c-454391b041d3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:26.341548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:26.342582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:26.342868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:26.343174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] PATH_INFO: `/v3/projects/f09ee72c9659421d9b89031b46720e23` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:26.347614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f09ee72c9659421d9b89031b46720e23)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:26.347614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:26.451011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f09ee72c9659421d9b89031b46720e23', 'request_id': 'req-00b823a4-512d-4209-ad3d-a6b5de7141c9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.456482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f09ee72c9659421d9b89031b46720e23', 'request_id': 'req-00b823a4-512d-4209-ad3d-a6b5de7141c9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.533759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f09ee72c9659421d9b89031b46720e23', 'request_id': 'req-00b823a4-512d-4209-ad3d-a6b5de7141c9'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.545882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] The token cache is being invalidate because project f09ee72c9659421d9b89031b46720e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:26.546171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.546801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.548365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 907/1805] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:26 2022] DELETE /identity/v3/projects/f09ee72c9659421d9b89031b46720e23 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:26.552725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:26.562292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69209a0a-386c-4a2f-a876-1c6b8487ec77 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:26.635111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:26.636552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:26.636847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:26.637133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] PATH_INFO: `/v3/users/916fb9390a1647a8a3a5817e49b047bc` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:26.638423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916fb9390a1647a8a3a5817e49b047bc)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:26.640422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:26.664786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:26.885828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916fb9390a1647a8a3a5817e49b047bc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.886422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916fb9390a1647a8a3a5817e49b047bc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.893058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916fb9390a1647a8a3a5817e49b047bc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.962312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916fb9390a1647a8a3a5817e49b047bc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.978350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e38c075f-5fc7-4415-aaeb-03969658b213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916fb9390a1647a8a3a5817e49b047bc'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:26.984559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 899/1806] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:26 2022] DELETE /identity/v3/users/916fb9390a1647a8a3a5817e49b047bc => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:27.383296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:27.385611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:27.410416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b823a4-512d-4209-ad3d-a6b5de7141c9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:27.474177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:27.475082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:27.475373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:27.475653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] PATH_INFO: `/v3/projects/9e9161e45f704df4b329cea38730848f` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:27.476910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9161e45f704df4b329cea38730848f)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:27.478909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:27.657089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9161e45f704df4b329cea38730848f', 'request_id': 'req-b100f264-ad60-4d77-9f20-f8ee6d0636e2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:27.662700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9161e45f704df4b329cea38730848f', 'request_id': 'req-b100f264-ad60-4d77-9f20-f8ee6d0636e2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:27.775695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9161e45f704df4b329cea38730848f', 'request_id': 'req-b100f264-ad60-4d77-9f20-f8ee6d0636e2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:27.788044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] The token cache is being invalidate because project 9e9161e45f704df4b329cea38730848f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:27.788332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:27.788994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b100f264-ad60-4d77-9f20-f8ee6d0636e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:27.790993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 908/1807] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:27 2022] DELETE /identity/v3/projects/9e9161e45f704df4b329cea38730848f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:27.796014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:27.796547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:27.796771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:27.816315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:27.817512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:27.877975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 900/1808] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:27.887389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:27.887687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:27.887976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:27.914763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:27.914763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:27.916649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 909/1809] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:27.920771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:27.931901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecbc60-04c9-4b99-8305-a82983075ac7 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:27.940345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:27.941047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:27.941461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:27.941728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:27.942471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:27.942867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:27.944622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:27.969149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 901/1810] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:27.973663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:27.984007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68483b-4817-4c85-a4cd-f9f179ad25dd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:27.988872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:27.989529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:27.989848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:27.990130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:27.990847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:27.992783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:27.997526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:28.122628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 910/1811] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:27 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:28.127047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.138554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee83bc6-1a24-42da-961f-bfffaac8ae04 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.212541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.213272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.213618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.213900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.214613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.216585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:28.317443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 902/1812] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:28 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:28.325331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.337140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11b03-70e9-429a-ae70-5cc2eab1b26c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.343377 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.344205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.344779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.345064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.346077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.347908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:28.372365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 911/1813] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:28.376199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.387333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4135c9-e8ce-4c4c-8969-ee0dd486eb96 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.392189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.392854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.393141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.393453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.394181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.395869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b3848-87fa-4ef1-ae3c-4da5639e7d35 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:28.407426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 903/1814] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:28.416140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.436580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2b43c-b195-43d8-bda6-fa5ee7a51d14 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.436580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.437071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.437393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.437675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] PATH_INFO: `/v3/projects/28413d9102f443649e6a1066bea4f759/users/947c1b5f8d1d4bec88050179e5b30389/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.459151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28413d9102f443649e6a1066bea4f759, user_id=947c1b5f8d1d4bec88050179e5b30389, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.462174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:28.676963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 912/1815] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:28 2022] PUT /identity/v3/projects/28413d9102f443649e6a1066bea4f759/users/947c1b5f8d1d4bec88050179e5b30389/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:28.688591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.688823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.689098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.695508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:28.717849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] MFA Rules not processed for user `947c1b5f8d1d4bec88050179e5b30389`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:28.719158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:28.788494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 904/1816] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:28.800766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.802633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:28.864625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec72f6-d4c0-4bfd-90fe-0454913185e1 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.868274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.868923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.869208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.869485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.877831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.879288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3834a7-891c-4d4f-8c08-5ae21c3ecc6a service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:28.888604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 913/1817] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:28.908061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:28.968840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c078fef-0a6d-4b60-986b-db28c0986e94 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:28.973864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:28.974769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:28.975108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:28.975415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] PATH_INFO: `/v3/users/947c1b5f8d1d4bec88050179e5b30389` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:28.976641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] RBAC: Authorizing `identity:delete_user(user_id=947c1b5f8d1d4bec88050179e5b30389)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:28.978654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:29.242786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947c1b5f8d1d4bec88050179e5b30389'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:29.244414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947c1b5f8d1d4bec88050179e5b30389'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:29.254550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947c1b5f8d1d4bec88050179e5b30389'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:29.328828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947c1b5f8d1d4bec88050179e5b30389'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:29.377918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947c1b5f8d1d4bec88050179e5b30389'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:29.392119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 905/1818] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:28 2022] DELETE /identity/v3/users/947c1b5f8d1d4bec88050179e5b30389 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:29.394511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:29.395322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:29.395906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:29.433761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:29.435025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:29.601595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 914/1819] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:29.618277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:29.620392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:29.648933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:29.743972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317ae07-ca1a-4d1d-ab45-6992108a362d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:29.749689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:29.750717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:29.751122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:29.751301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:29.761856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:29.763492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ac3de-9543-4ecd-bfc6-0791eaab5175 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:29.775392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 906/1820] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:30.595744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:30.597278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:30.610507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff22be2-7f5c-44d9-ae57-ed40408ef965 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:30.615820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:30.616815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:30.617098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:30.617374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] PATH_INFO: `/v3/projects/28413d9102f443649e6a1066bea4f759` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:30.618728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28413d9102f443649e6a1066bea4f759)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:30.629609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:30.815444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28413d9102f443649e6a1066bea4f759', 'request_id': 'req-5d057159-225e-4344-8b92-a5525db19621'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:30.816796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:30.822468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28413d9102f443649e6a1066bea4f759', 'request_id': 'req-5d057159-225e-4344-8b92-a5525db19621'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:30.964410 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28413d9102f443649e6a1066bea4f759', 'request_id': 'req-5d057159-225e-4344-8b92-a5525db19621'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:30.976137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] The token cache is being invalidate because project 28413d9102f443649e6a1066bea4f759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:30.976429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:30.977065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5d057159-225e-4344-8b92-a5525db19621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:30.978728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 915/1821] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:30 2022] DELETE /identity/v3/projects/28413d9102f443649e6a1066bea4f759 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:30.983814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:30.984356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:30.984702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.003453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:31.004831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:31.065711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 907/1822] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:31.077654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.077654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.077654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.113745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:31.115062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:31.124306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 916/1823] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:31.128967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.139975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67f16-339c-413c-9ceb-e85bc18601e0 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.146665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.146665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.146665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.146665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.147017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:31.147017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.148554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.164933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 908/1824] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:31.166241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.180472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95213d8d-5deb-4c6a-aae2-9c0a58d9a819 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.191219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.191992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.192515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.192735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.194025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.196569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.272162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 917/1825] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:31 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:31.276567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.287629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca216608-5d26-4fa4-bc37-e28a2cf5cc5c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.369311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.370700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.370870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.371194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.371902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.374574 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.552852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 909/1826] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:31 2022] POST /identity/v3/users => generated 287 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:44:31.562463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.572800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc677fa4-a839-4cce-86b3-b319133d9d46 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.577249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.577927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.578208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.578485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.579244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.580815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.593972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 918/1827] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:31.596609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.606787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de6594-591c-41b9-a43a-a37e5de8b569 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.611375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.612021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.612308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.612587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.613314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.614893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af925d-a05b-4a52-b4a9-23fecfa874e8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.624930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 910/1828] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:31.634035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:31.643888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d7f37-4fc7-43e9-bb2c-1f90e58b744d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:31.648372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:31.649242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.649528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.649807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] PATH_INFO: `/v3/projects/6bbe00a9e4e840b4bdc855c23bf81053/users/669d8f2d9a4f46599727f16bb42c4405/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.664820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbe00a9e4e840b4bdc855c23bf81053, user_id=669d8f2d9a4f46599727f16bb42c4405, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:31.668064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:31.886153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 919/1829] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:31 2022] PUT /identity/v3/projects/6bbe00a9e4e840b4bdc855c23bf81053/users/669d8f2d9a4f46599727f16bb42c4405/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:31.897531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:31.897841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:31.898117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:31.925782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] MFA Rules not processed for user `669d8f2d9a4f46599727f16bb42c4405`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:31.927043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:31.990608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 911/1830] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:32.001941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:32.003664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:32.058882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f70d5-9b08-4be5-ab82-4901ee28ce40 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:32.062432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:32.063113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:32.063402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:32.063682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:32.072361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:32.074329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dcb31-de1d-4583-a99e-65cd583b0990 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:32.083066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 920/1831] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:32.102890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:32.164786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4fcc-8390-47aa-9ad8-0e02d49c5340 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:32.169283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:32.170148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:32.170431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:32.170736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] PATH_INFO: `/v3/users/669d8f2d9a4f46599727f16bb42c4405` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:32.171965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=669d8f2d9a4f46599727f16bb42c4405)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:32.173916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:32.239108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:32.395117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669d8f2d9a4f46599727f16bb42c4405'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:32.395907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669d8f2d9a4f46599727f16bb42c4405'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:32.402839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669d8f2d9a4f46599727f16bb42c4405'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:32.493632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669d8f2d9a4f46599727f16bb42c4405'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:32.523253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669d8f2d9a4f46599727f16bb42c4405'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:32.529334 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 912/1832] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:32 2022] DELETE /identity/v3/users/669d8f2d9a4f46599727f16bb42c4405 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:32.534168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:32.534826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:32.535283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:32.556828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:32.556828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:32.628566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 921/1833] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:32.638015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:32.639535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:32.705833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302d9b7-2d54-4e3e-a06a-cfb6eb8b3b9c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:32.709449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:32.710094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:32.710379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:32.710657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:32.719523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:32.720952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868a9d9-744c-4cdf-9212-f5020dc327d5 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:32.731831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 913/1834] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:32.771215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:33.226252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:33.227754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:33.242027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b7d0d-5ae1-4035-bf30-8bbd0ec2b264 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:33.245697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:33.246607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.246977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.247264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] PATH_INFO: `/v3/projects/6bbe00a9e4e840b4bdc855c23bf81053` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.248469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbe00a9e4e840b4bdc855c23bf81053)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:33.250380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:33.262764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:33.381035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbe00a9e4e840b4bdc855c23bf81053', 'request_id': 'req-ede270e9-bbb4-44a3-a20d-d62787fcff63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:33.387276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbe00a9e4e840b4bdc855c23bf81053', 'request_id': 'req-ede270e9-bbb4-44a3-a20d-d62787fcff63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:33.530018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbe00a9e4e840b4bdc855c23bf81053', 'request_id': 'req-ede270e9-bbb4-44a3-a20d-d62787fcff63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:33.550151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] The token cache is being invalidate because project 6bbe00a9e4e840b4bdc855c23bf81053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:33.550752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:33.552185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-ede270e9-bbb4-44a3-a20d-d62787fcff63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:33.554441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 922/1835] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:33 2022] DELETE /identity/v3/projects/6bbe00a9e4e840b4bdc855c23bf81053 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:33.562694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.563258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.563258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.586993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:33.588219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:33.655052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 914/1836] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:33.663019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.663480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.663781 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.685493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:33.687015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:33.695881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 923/1837] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:33.709091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:33.716621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e51a-b3d7-4b31-a972-2288e9da9a86 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:33.721962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:33.722623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.722905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.723215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.723973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:33.724288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:33.727149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:33.740889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 915/1838] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:44:33.745606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:33.776333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9c9a5-a229-449d-bad1-5bdfacf6e853 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:33.784965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:33.786269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.788025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.788025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.788859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:33.791322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:33.909180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 924/1839] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:33 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:33.916747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:33.927151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12ee46-3a7a-4f2d-a058-032ab38920e9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:33.988364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:33.989036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:33.989320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:33.989725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:33.990456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:33.992637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:34.093927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 916/1840] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:33 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:34.099669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:34.112186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c267-6c13-40be-92ce-4a672e991dd9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:34.117428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:34.118186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:34.118563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:34.118911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:34.119758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:34.121906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:34.132703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 925/1841] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:34.137029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:34.147370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27bfd2-7b66-42f5-9003-782e78bccd3b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:34.152201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:34.152920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:34.153211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:34.153490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:34.154226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:34.156000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ce7c8-52bf-477c-a850-c2ae7fdb8a12 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:34.174739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:34.175310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 917/1842] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:34.192524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c0bc-bc1e-4027-8b49-c942a9b9b890 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:34.198032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:34.199247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:34.199611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:34.199954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] PATH_INFO: `/v3/projects/6b33cf919b594022810e0c9c44f91384/users/b83d15e055b843c38e4b8e0fee263c4c/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:34.218097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b33cf919b594022810e0c9c44f91384, user_id=b83d15e055b843c38e4b8e0fee263c4c, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:34.221574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:34.330580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 926/1843] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:34 2022] PUT /identity/v3/projects/6b33cf919b594022810e0c9c44f91384/users/b83d15e055b843c38e4b8e0fee263c4c/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:34.341325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60135b80-906c-45c0-b74f-490cb9d73ac5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:34.341618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60135b80-906c-45c0-b74f-490cb9d73ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:34.341953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60135b80-906c-45c0-b74f-490cb9d73ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:34.368035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-60135b80-906c-45c0-b74f-490cb9d73ac5 None None] MFA Rules not processed for user `b83d15e055b843c38e4b8e0fee263c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:34.369252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-60135b80-906c-45c0-b74f-490cb9d73ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:34.399400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:34.445239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 918/1844] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:34.457248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:34.458975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:34.517119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d31cf-dc99-4769-8939-c7e69e28e3df admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:34.520745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:34.521397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:34.521688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:34.521998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:34.530395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:34.531862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2f150-0233-4ec8-8cb6-284a00c760ac service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:34.540705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 927/1845] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:35.597762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:35.598283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:35.598283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:35.599848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 919/1846] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:44:35.630542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:35.631125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:35.631125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:35.663328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:35.664685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:35.701847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:35.736845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 928/1847] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:44:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:35.773709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:35.783496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:35.813675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47891cd0-5fb5-4b7b-80fe-079232bc9709 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:35.834658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:35.835438 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:35.835630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:35.835861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:35.846633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:35.848269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a9f5-0135-4d45-ab47-16dc76ae9a39 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:35.873481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 920/1848] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:35.999507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:36.005152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:36.115340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d81be-edce-494b-88f1-da17ff9d5526 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:36.127859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:36.129941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:36.141272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:36.141272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] PATH_INFO: `/v3/users/2267e41c80c84e9eb463fab2d08723c2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:36.141272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2267e41c80c84e9eb463fab2d08723c2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:36.142601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:36.389971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2267e41c80c84e9eb463fab2d08723c2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:36.390688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2267e41c80c84e9eb463fab2d08723c2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:36.405966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2267e41c80c84e9eb463fab2d08723c2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:36.483983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2267e41c80c84e9eb463fab2d08723c2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:36.515539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2267e41c80c84e9eb463fab2d08723c2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:36.524250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 929/1849] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:35 2022] DELETE /identity/v3/users/2267e41c80c84e9eb463fab2d08723c2 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:36.528729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:36.529147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:36.529491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:36.552551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:36.554006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:36.636326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 921/1850] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:36.654777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:36.656249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:36.739021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d6be3-4199-4af0-a873-7a598498b40d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:36.742851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:36.743603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:36.743917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:36.744223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:36.750708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:36.754580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:36.756047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:36.766356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 930/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:37.260517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:37.262424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:37.278654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d73ef-e47f-4164-b3fb-d5807913c854 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:37.287509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:37.288467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:37.288819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:37.289296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] PATH_INFO: `/v3/projects/5fe350abc5364ceab16d16f9e78d35ba` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:37.290648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe350abc5364ceab16d16f9e78d35ba)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:37.292761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:37.428190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe350abc5364ceab16d16f9e78d35ba', 'request_id': 'req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.433936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe350abc5364ceab16d16f9e78d35ba', 'request_id': 'req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.518520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe350abc5364ceab16d16f9e78d35ba', 'request_id': 'req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.531462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] The token cache is being invalidate because project 5fe350abc5364ceab16d16f9e78d35ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:37.531899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.532670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.534640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 922/1852] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:37 2022] DELETE /identity/v3/projects/5fe350abc5364ceab16d16f9e78d35ba => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:37.538877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:37.551892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c68d47-150b-4a37-a2af-a1d615fdebb6 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:37.619236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:37.622424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:37.622424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:37.622424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] PATH_INFO: `/v3/users/2310d9c6393b4b9d806547ddb9341396` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:37.622424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2310d9c6393b4b9d806547ddb9341396)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:37.624389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:37.837626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2310d9c6393b4b9d806547ddb9341396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.838328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2310d9c6393b4b9d806547ddb9341396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.845398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2310d9c6393b4b9d806547ddb9341396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.935656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2310d9c6393b4b9d806547ddb9341396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.956096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-42f2758a-ec0a-400e-8958-a021694e85f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2310d9c6393b4b9d806547ddb9341396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:37.963948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 931/1853] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:37 2022] DELETE /identity/v3/users/2310d9c6393b4b9d806547ddb9341396 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:38.399969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:38.402474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:38.415583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12d79d-3e6d-4d71-9b73-c40b4dc1774b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:38.474520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:38.501832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:38.502807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:38.503212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:38.503801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] PATH_INFO: `/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:38.504877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbd2d63f4ec44388bf73c9ea0c47877)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:38.506864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:38.661409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbd2d63f4ec44388bf73c9ea0c47877', 'request_id': 'req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:38.668607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbd2d63f4ec44388bf73c9ea0c47877', 'request_id': 'req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:38.794408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbd2d63f4ec44388bf73c9ea0c47877', 'request_id': 'req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:38.810042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] The token cache is being invalidate because project 5dbd2d63f4ec44388bf73c9ea0c47877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:38.810286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:38.811506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9054bf3d-71ad-4d43-95ba-5bc3ee80c8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:38.813419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 923/1854] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:38 2022] DELETE /identity/v3/projects/5dbd2d63f4ec44388bf73c9ea0c47877 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:38.817956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:38.818275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:38.818562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:38.837554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:38.839308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:38.898684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 932/1855] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:38.908597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:38.909013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:38.909357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:38.928485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:38.929803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:38.938915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 924/1856] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:38.943168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:38.953333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe980f-8a96-4cd9-9c60-8ec781aadab4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:38.958151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:38.958817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:38.959145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:38.959431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:38.960169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:38.960483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:38.962094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:38.972106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:38.975442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 933/1857] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:38.980548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:38.992847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934c6b6-b97b-4ff2-bf01-6c141a5768a8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:38.998900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:38.999805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.000162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.000508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.001278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.003395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.114254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 925/1858] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:38 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:39.118261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:39.128393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b660bf-2937-47fe-981d-a7a9a2ac04c6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:39.184582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:39.185263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.185551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.185833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.186535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.188506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.296530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 934/1859] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:39 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:39.300279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:39.310551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56447-391d-4ce2-b954-fad133eaa235 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:39.315599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:39.316372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.316779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.317517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.317987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.327991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.331790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 926/1860] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:39.338806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:39.346904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85e831-3a05-4dc4-81a4-9e1563629a38 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:39.353196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:39.353533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.353813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.354102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.354836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.356474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3ac0-2ae9-4df6-888d-ae335e807bf1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.367637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 935/1861] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:39.371782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:39.381628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34524fb6-5156-457b-8420-41dc97019a15 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:39.386679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:39.387609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.387892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.388168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] PATH_INFO: `/v3/projects/5102351f409342958acccdaa97aef2b1/users/69a7c66fed71473aaa038a8ec9c5c7fd/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.402593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5102351f409342958acccdaa97aef2b1, user_id=69a7c66fed71473aaa038a8ec9c5c7fd, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.405274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.496127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 927/1862] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:39 2022] PUT /identity/v3/projects/5102351f409342958acccdaa97aef2b1/users/69a7c66fed71473aaa038a8ec9c5c7fd/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:39.506330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.506628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.506909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.532973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] MFA Rules not processed for user `69a7c66fed71473aaa038a8ec9c5c7fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:39.534176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:39.647359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 936/1863] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:39.661492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:39.663333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:39.732718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee7d1-5f86-41bc-aa7c-f8d57d0e8fa8 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:39.736598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:39.737261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:39.737548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:39.737827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:39.747614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:39.749056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234bef4-0a08-47ba-9486-cd44c865f691 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:39.758454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 928/1864] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:44.502903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:44.502903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:44.642125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85c673-73b1-4f5b-afc6-b6dcb97727e2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:44.647896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:44.648857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:44.649240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:44.649628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] PATH_INFO: `/v3/users/b83d15e055b843c38e4b8e0fee263c4c` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:44.652312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83d15e055b843c38e4b8e0fee263c4c)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:44.653551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:44.666008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:44.983889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83d15e055b843c38e4b8e0fee263c4c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:44.985278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83d15e055b843c38e4b8e0fee263c4c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:45.005113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83d15e055b843c38e4b8e0fee263c4c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:45.112108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83d15e055b843c38e4b8e0fee263c4c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:45.126514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83d15e055b843c38e4b8e0fee263c4c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:45.133348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 937/1865] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:44 2022] DELETE /identity/v3/users/b83d15e055b843c38e4b8e0fee263c4c => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:45.138324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:45.138815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:45.138979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:45.151484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:45.159554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:45.160771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:45.234312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 929/1866] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:45.243938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:45.245944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:45.309053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba437ad-44d7-4563-b216-42437e1ecad8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:45.312854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:45.313518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:45.313803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:45.314080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:45.325050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:45.326563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa612db-143d-4b3d-8a87-ac9699acbf98 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:45.336819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 938/1867] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:46.017213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:46.018753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:46.031271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1d871-2e1b-440e-837f-552bc631bf40 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:46.036070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:46.036941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.037256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.037606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] PATH_INFO: `/v3/projects/6b33cf919b594022810e0c9c44f91384` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.039201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b33cf919b594022810e0c9c44f91384)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:46.041397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:46.170772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b33cf919b594022810e0c9c44f91384', 'request_id': 'req-865618b5-a961-41b1-996c-496696e5385a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:46.176310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b33cf919b594022810e0c9c44f91384', 'request_id': 'req-865618b5-a961-41b1-996c-496696e5385a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:46.266318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b33cf919b594022810e0c9c44f91384', 'request_id': 'req-865618b5-a961-41b1-996c-496696e5385a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:46.277882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] The token cache is being invalidate because project 6b33cf919b594022810e0c9c44f91384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:46.278173 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:46.278802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-865618b5-a961-41b1-996c-496696e5385a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:46.280367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 930/1868] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:46 2022] DELETE /identity/v3/projects/6b33cf919b594022810e0c9c44f91384 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:46.285176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.285629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.285740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.303551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:46.304737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:46.373568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 939/1869] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:46.382542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.382841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.383171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.478987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:46.478987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:46.478987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 931/1870] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:46.480363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:46.480363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963ca4-c6e3-4636-8ab3-3f6ac3f82d26 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:46.480363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:46.480363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:46.480970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 940/1871] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:46.482628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:46.491892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9ad4b-2882-4715-9cc8-a5906a8071c5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:46.496865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:46.497515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.497803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.498082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.498778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:46.500848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:46.766205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 932/1872] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:46 2022] POST /identity/v3/projects => generated 365 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:46.770463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:46.781001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce7f5-a269-4ef2-9043-d0ce69122a41 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:46.795529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:46.838882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:46.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:46.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:46.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:46.840953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:46.842752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:46.971139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 941/1873] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:46 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:46.974461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:46.991865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0276448-0154-4cef-9fdf-feff9b84864f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:46.998711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:46.999865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:47.000313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:47.000791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:47.002020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:47.003985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:47.028881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 933/1874] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:47.029494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:47.046829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8690-af92-40ed-b885-e23e17e50a6d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:47.053208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:47.053208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:47.053208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:47.053481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:47.054182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:47.055996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddd1df-1ecf-4a70-b7b1-68b7d3a3fca5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:47.073120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 942/1875] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:47.074631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:47.086483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e0b4-6471-4e03-88a5-a8944a771bd0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:47.094455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:47.095383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:47.095937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:47.096184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] PATH_INFO: `/v3/projects/4f2c3307a1e94d99a5f6426aefbdd8f8/users/8913d20bc0794f29afc75dd7300903e0/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:47.121002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2c3307a1e94d99a5f6426aefbdd8f8, user_id=8913d20bc0794f29afc75dd7300903e0, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:47.123920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:47.258963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 934/1876] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:47 2022] PUT /identity/v3/projects/4f2c3307a1e94d99a5f6426aefbdd8f8/users/8913d20bc0794f29afc75dd7300903e0/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:47.269207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07979749-5167-4554-b119-a203401c74ae None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:47.269503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07979749-5167-4554-b119-a203401c74ae None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:47.269791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07979749-5167-4554-b119-a203401c74ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:47.296766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-07979749-5167-4554-b119-a203401c74ae None None] MFA Rules not processed for user `8913d20bc0794f29afc75dd7300903e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:47.297983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-07979749-5167-4554-b119-a203401c74ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:47.361141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 943/1877] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:47.372340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:47.377495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:47.407823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:47.433990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c99b-90f1-491d-89cf-a80c301d31a3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:47.437662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:47.438316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:47.438604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:47.438886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:47.447461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:47.448918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:47.457587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 935/1878] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:51.300498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07979749-5167-4554-b119-a203401c74ae None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:51.302006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-07979749-5167-4554-b119-a203401c74ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:51.469692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07979749-5167-4554-b119-a203401c74ae None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:51.474704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:51.475818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:51.477249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:51.477721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] PATH_INFO: `/v3/users/c0037f95e6cd478283e1f44dd1b1b7f0` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:51.479207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0037f95e6cd478283e1f44dd1b1b7f0)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:51.481964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:51.609005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:51.803439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0037f95e6cd478283e1f44dd1b1b7f0'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:51.805357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0037f95e6cd478283e1f44dd1b1b7f0'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:51.814075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0037f95e6cd478283e1f44dd1b1b7f0'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:51.847190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:51.848745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:51.868793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614a559-e8de-4209-a7cf-aa2ff2d897fa service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:51.868793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:51.868793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:51.868793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:51.869175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] PATH_INFO: `/v3/users/8b1e43e5d56f4085af872e7c794dcb9b` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:51.879855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1e43e5d56f4085af872e7c794dcb9b)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:51.882012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:51.888916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0037f95e6cd478283e1f44dd1b1b7f0'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:51.911407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc797bab-c078-4546-a187-58ac79ac030b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0037f95e6cd478283e1f44dd1b1b7f0'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:51.918724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 944/1879] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:51 2022] DELETE /identity/v3/users/c0037f95e6cd478283e1f44dd1b1b7f0 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:51.924445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:51.924797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:51.925365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:51.947787 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:51.952642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:52.057115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 945/1880] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:52.067676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:52.069245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:52.139707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f33dd-3da8-4903-8ba9-e4b73f557380 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:52.143981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:52.144717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:52.145076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:52.145415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:52.155971 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:52.157665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce773018-6cbc-4491-a492-cc1650b6cd8e service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:52.170818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 946/1881] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:52.227593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1e43e5d56f4085af872e7c794dcb9b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:52.236613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1e43e5d56f4085af872e7c794dcb9b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:52.260328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1e43e5d56f4085af872e7c794dcb9b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:52.447374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1e43e5d56f4085af872e7c794dcb9b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:52.485378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1e43e5d56f4085af872e7c794dcb9b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:52.509951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 936/1882] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:51 2022] DELETE /identity/v3/users/8b1e43e5d56f4085af872e7c794dcb9b => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:52.511818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:52.512267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:52.512538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:52.539571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:52.541321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:52.611697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:52.636855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 947/1883] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:52.656491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:52.659425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:52.739548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485499-d7d6-4b8f-b149-1f5fce374e7e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:52.744416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:52.745080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:52.745489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:52.745661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:52.756220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:52.757644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:52.768137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 937/1884] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:52.928670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:52.930101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:52.957650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55749b-86d2-4901-8f1c-e7bc646e0189 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:52.971624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:52.972636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:52.972824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:52.973071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] PATH_INFO: `/v3/projects/1e51131da4ff475ab3fae4fa3b2b3b42` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:52.974429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e51131da4ff475ab3fae4fa3b2b3b42)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:52.976416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:53.186021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e51131da4ff475ab3fae4fa3b2b3b42', 'request_id': 'req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.192317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e51131da4ff475ab3fae4fa3b2b3b42', 'request_id': 'req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.356046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e51131da4ff475ab3fae4fa3b2b3b42', 'request_id': 'req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.376306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] The token cache is being invalidate because project 1e51131da4ff475ab3fae4fa3b2b3b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:53.376604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.387534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.392065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 948/1885] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:52 2022] DELETE /identity/v3/projects/1e51131da4ff475ab3fae4fa3b2b3b42 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:53.393258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:53.410638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f31d-ff50-4c82-97a7-c3b79c638be2 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:53.471018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:53.486264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:53.487289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:53.487552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:53.487828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] PATH_INFO: `/v3/users/7b2844d455b5442e8afe38fb7bb5b21f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:53.489018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2844d455b5442e8afe38fb7bb5b21f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:53.490950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:53.637306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:53.638780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:53.657504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8049a-e0ed-4745-8a00-ac51cabbd7ff admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:53.663175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:53.667515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:53.667515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:53.667515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] PATH_INFO: `/v3/projects/a5742f8a1bc94594ae52ac814bc306cf` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:53.667515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5742f8a1bc94594ae52ac814bc306cf)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:53.668442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:53.958528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2844d455b5442e8afe38fb7bb5b21f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.959228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2844d455b5442e8afe38fb7bb5b21f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.965529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5742f8a1bc94594ae52ac814bc306cf', 'request_id': 'req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.967385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2844d455b5442e8afe38fb7bb5b21f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:53.971489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5742f8a1bc94594ae52ac814bc306cf', 'request_id': 'req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.066619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2844d455b5442e8afe38fb7bb5b21f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.079613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2844d455b5442e8afe38fb7bb5b21f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.085707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 938/1886] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:53 2022] DELETE /identity/v3/users/7b2844d455b5442e8afe38fb7bb5b21f => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:54.247567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5742f8a1bc94594ae52ac814bc306cf', 'request_id': 'req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.259185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] The token cache is being invalidate because project a5742f8a1bc94594ae52ac814bc306cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:54.259514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.260189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.262444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 949/1887] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:53 2022] DELETE /identity/v3/projects/a5742f8a1bc94594ae52ac814bc306cf => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:54.266489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:54.268002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:54.334394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1c698-f76f-4060-b225-3bf550246d41 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:54.339021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:54.340049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:54.340210 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:54.340473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] PATH_INFO: `/v3/users/89ed13b30b714e0ebfaef87f65aec581` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:54.342246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ed13b30b714e0ebfaef87f65aec581)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:54.343805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:54.500071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:54.501433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:54.585013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:54.600785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69becd3-4788-4e9d-b5a7-9b775f8e9aff admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:54.608598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:54.608598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:54.608598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:54.608598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] PATH_INFO: `/v3/users/69a7c66fed71473aaa038a8ec9c5c7fd` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:54.609041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a7c66fed71473aaa038a8ec9c5c7fd)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:54.610763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:54.610898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ed13b30b714e0ebfaef87f65aec581'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.611606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ed13b30b714e0ebfaef87f65aec581'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.619687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ed13b30b714e0ebfaef87f65aec581'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.765439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ed13b30b714e0ebfaef87f65aec581'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.787223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ed13b30b714e0ebfaef87f65aec581'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.796174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 939/1888] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:54 2022] DELETE /identity/v3/users/89ed13b30b714e0ebfaef87f65aec581 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:54.797551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:54.799223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:54.877869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac301-6586-44b1-b556-d4224bab0c91 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:54.888746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:54.890362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:54.890652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:54.890954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] PATH_INFO: `/v3/projects/5ca390c9869741e0b58ab69552308c36` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:54.892612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca390c9869741e0b58ab69552308c36)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:54.894630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:54.957761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a7c66fed71473aaa038a8ec9c5c7fd'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.958437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a7c66fed71473aaa038a8ec9c5c7fd'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:54.968370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a7c66fed71473aaa038a8ec9c5c7fd'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.045785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca390c9869741e0b58ab69552308c36', 'request_id': 'req-e0867d8b-8223-41c4-8214-3c04638b57f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.052899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca390c9869741e0b58ab69552308c36', 'request_id': 'req-e0867d8b-8223-41c4-8214-3c04638b57f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.055120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a7c66fed71473aaa038a8ec9c5c7fd'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.078153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5a2a09f7-7553-4882-ac48-4367eb095c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a7c66fed71473aaa038a8ec9c5c7fd'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.087254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 950/1889] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:54 2022] DELETE /identity/v3/users/69a7c66fed71473aaa038a8ec9c5c7fd => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:55.091456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.092053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.093126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.116482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:55.117891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:55.181527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca390c9869741e0b58ab69552308c36', 'request_id': 'req-e0867d8b-8223-41c4-8214-3c04638b57f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.196624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] The token cache is being invalidate because project 5ca390c9869741e0b58ab69552308c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:55.198992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.198992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e0867d8b-8223-41c4-8214-3c04638b57f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.200181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 940/1890] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:54 2022] DELETE /identity/v3/projects/5ca390c9869741e0b58ab69552308c36 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:55.206797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e879db-b868-4d9c-97fa-1a4f91f5dd4d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.207358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e879db-b868-4d9c-97fa-1a4f91f5dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.208479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e879db-b868-4d9c-97fa-1a4f91f5dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.211807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 951/1891] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:55.220318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:55.221784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:55.237540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c9e879db-b868-4d9c-97fa-1a4f91f5dd4d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:55.239125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c9e879db-b868-4d9c-97fa-1a4f91f5dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:55.250397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 941/1892] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:55.270677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.271722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.272086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.295528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e6952-24c5-479a-9cb3-8c77a725cf79 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:55.297783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:55.300312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:55.300864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:55.301716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.302203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.302630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.311662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 942/1893] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:55.315517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:55.315645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:55.318394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:55.331886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124316f-bfd2-490e-9863-6470520a9864 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:55.337613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:55.338955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.339326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.339705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.341290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:55.341798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:55.344040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 952/1894] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:44:55.345625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:55.370271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 943/1895] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:55.374974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:55.385912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c4626-31ef-49c5-847b-e8a8fc4d9ff4 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:55.392925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:55.393966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.394435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.394885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.395822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:55.398683 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:55.499123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:55.668353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 953/1896] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:55 2022] POST /identity/v3/projects => generated 383 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:55.668636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:55.669703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:55.670992 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:55.684238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540003e-65ce-4603-b455-b29be2adf47f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:55.759156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:55.760671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.761105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.761448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] PATH_INFO: `/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.763479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0971b1cc698f44d4b8ee15fdf4f38df4)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:55.766161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:55.777703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00639-8bec-4014-aa61-dacb5abce5e5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:55.782962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:55.783794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:55.784182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:55.785204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:55.785982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:55.788221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:55.934214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0971b1cc698f44d4b8ee15fdf4f38df4', 'request_id': 'req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.939971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0971b1cc698f44d4b8ee15fdf4f38df4', 'request_id': 'req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:55.986398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 954/1897] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:55 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:55.988239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.000808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe8d-96b5-42f9-979a-ff698c86e256 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.074679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.074679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.074679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.074679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.075878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.075878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.076626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 955/1898] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:56.077656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.079480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.091938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0237-6d24-466b-b8c4-85bb1434b86b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.096443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.097582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.097870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.098156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] PATH_INFO: `/v3/projects/5102351f409342958acccdaa97aef2b1` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.102175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0971b1cc698f44d4b8ee15fdf4f38df4', 'request_id': 'req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.102301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5102351f409342958acccdaa97aef2b1)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.102301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.111880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] The token cache is being invalidate because project 0971b1cc698f44d4b8ee15fdf4f38df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:56.112172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.112809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.114447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 944/1899] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:55 2022] DELETE /identity/v3/projects/0971b1cc698f44d4b8ee15fdf4f38df4 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:56.115433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.116814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.130171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbf4ce-aaf1-42bc-96c0-c0f4791fc941 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.135074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.135769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.136054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.136335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.137070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.138706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f79f8-88dc-4941-a51e-50c08ea1d0ea admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.149460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 945/1900] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:56.150716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.151027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.151335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.169718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:56.170905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.178684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 946/1901] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:56.179500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.188723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b160-3ac1-4750-ad20-574d94005d76 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.193890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.195046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.199932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.199932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] PATH_INFO: `/v3/projects/09769d07226d4816806af14f289cc49f/users/752b8e20f97f42f7b185972a7c9f9f65/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.211265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09769d07226d4816806af14f289cc49f, user_id=752b8e20f97f42f7b185972a7c9f9f65, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.215312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdabce4-93bb-4fac-8583-5fb5383a6d10 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.295621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 947/1902] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:56 2022] PUT /identity/v3/projects/09769d07226d4816806af14f289cc49f/users/752b8e20f97f42f7b185972a7c9f9f65/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:56.296928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203e991-0c85-43d4-85dd-d54da99aefe7 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.297246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203e991-0c85-43d4-85dd-d54da99aefe7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.297528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203e991-0c85-43d4-85dd-d54da99aefe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.312868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5102351f409342958acccdaa97aef2b1', 'request_id': 'req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.315985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c203e991-0c85-43d4-85dd-d54da99aefe7 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:56.317204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c203e991-0c85-43d4-85dd-d54da99aefe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.318878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5102351f409342958acccdaa97aef2b1', 'request_id': 'req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.377338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 948/1903] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:56.378596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba240-bf39-4d93-aa82-2217eba483f0 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.378891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba240-bf39-4d93-aa82-2217eba483f0 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.379208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba240-bf39-4d93-aa82-2217eba483f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.392512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:56.406467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a01ba240-bf39-4d93-aa82-2217eba483f0 None None] MFA Rules not processed for user `752b8e20f97f42f7b185972a7c9f9f65`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:56.407721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a01ba240-bf39-4d93-aa82-2217eba483f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.420480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5102351f409342958acccdaa97aef2b1', 'request_id': 'req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.436139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] The token cache is being invalidate because project 5102351f409342958acccdaa97aef2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:56.442128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.442128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:56.442128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 956/1904] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:56 2022] DELETE /identity/v3/projects/5102351f409342958acccdaa97aef2b1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:56.442128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.442128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.454872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd695f-c13c-4f36-9a64-7520d0dcaf40 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.459853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.460625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.460802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.461088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.461820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:56.462135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.463775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.473420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 949/1905] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:56.474824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.475169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.475451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.479159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 957/1906] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:56.480262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.483476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.496433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:56.496695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986764-0ab6-439e-b375-bddafc5b0ac3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.498445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.501346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.502012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.502304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.502584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.503326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.505353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5f7af-753c-4fc7-aa50-89bbcf8a27c6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.506697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 950/1907] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:56.507320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.517522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de448a-64ea-4e95-87de-d585344a8b10 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.522368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.523063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.523351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.523628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.524611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.526555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.640261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 958/1908] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:56.641570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.641864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.642143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.660246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:56.661469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:56.702399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 951/1909] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:56.703257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.733425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 959/1910] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:56.733425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:56.744287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff7de-0cdb-487c-ae12-3b2073df5cff None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.780949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e21cf-c9d7-4907-b413-bc94e35abc4b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:56.786724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.787433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.787790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.787963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.788882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.790771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.820829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:56.821529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:56.821823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:56.822108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:56.822872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:56.825403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:56.999863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 960/1911] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:57.001283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 952/1912] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:56 2022] POST /identity/v3/users => generated 300 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:57.001419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.004370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.013184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a649f-7f8d-4595-ad59-62b3329cdfce admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.016062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026f5a5-0df6-46ea-8584-73d9da9676dd admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.018831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.019581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.019873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.020161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.020864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.021005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:44:57.021199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.021516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.021802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.022082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.022813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.022958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.024439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.034600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 953/1913] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:57.035674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.037182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 961/1914] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:44:57.042519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.045484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3b344-6890-4cb1-9811-2a59dabe2280 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.050414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.051126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.051417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.051701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.052439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.052544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01545bc5-a42b-49f9-b5d0-950d635ca8f0 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.054067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.057268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.058654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.059142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.059425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.060138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.062041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.065777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 954/1915] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:57.066610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.075606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:57.080019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6fa5-b778-4cee-8fac-e7a82accae1f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.084878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.085546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.085829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.086104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.086833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.088497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.099046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 955/1916] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:57.099849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.111445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2edf6f-5a55-4b68-b1e7-166766266f96 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.116154 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.117041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.117318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.117596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] PATH_INFO: `/v3/projects/a7411eec00a3451399a785ebdfe320ae/users/8eb22f7aeb0f417f962b8ba4bfc9cea5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.132919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7411eec00a3451399a785ebdfe320ae, user_id=8eb22f7aeb0f417f962b8ba4bfc9cea5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.135629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.211485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 962/1917] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:57 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:44:57.212442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.274552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb23404-3d02-4ad4-bd58-7e2d384d1047 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.279672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.280606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.280893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.281180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] PATH_INFO: `/v3/projects/d844855fe5c44ec19d4102f0b6d08ffd/users/6cf8849f2b4d408695d73a6b55bc5632/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.296926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d844855fe5c44ec19d4102f0b6d08ffd, user_id=6cf8849f2b4d408695d73a6b55bc5632, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.299157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 956/1918] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:57 2022] PUT /identity/v3/projects/a7411eec00a3451399a785ebdfe320ae/users/8eb22f7aeb0f417f962b8ba4bfc9cea5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:57.299796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.300009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.362830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a815753-1803-4ba7-bfc8-e154ed2eaaf2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.367738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.368743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.369039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.369339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.370065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.372884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.436247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 963/1919] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:57 2022] PUT /identity/v3/projects/d844855fe5c44ec19d4102f0b6d08ffd/users/6cf8849f2b4d408695d73a6b55bc5632/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:57.437263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.496569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c260-3bd8-4e22-9157-c3fd159f1da3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.501735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.502413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.502689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.503005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.503731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.505327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bde0e7-ca56-4ad9-89f3-7adeb58d4a94 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.515202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 964/1920] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:57.516593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.516881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.517164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.542172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] MFA Rules not processed for user `6cf8849f2b4d408695d73a6b55bc5632`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:57.543843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:57.619752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 965/1921] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:57.619752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.629211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae05896-ce88-40f3-aa72-a1b9a87b677a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.635675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.636644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.637170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.637459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] PATH_INFO: `/v3/projects/a7411eec00a3451399a785ebdfe320ae/users/8eb22f7aeb0f417f962b8ba4bfc9cea5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.639506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7411eec00a3451399a785ebdfe320ae, user_id=8eb22f7aeb0f417f962b8ba4bfc9cea5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.642162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.654125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 966/1922] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:57 2022] PUT /identity/v3/projects/a7411eec00a3451399a785ebdfe320ae/users/8eb22f7aeb0f417f962b8ba4bfc9cea5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:57.654864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.659178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 957/1923] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:57 2022] POST /identity/v3/users => generated 288 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:57.659975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.673380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e103a-5eaa-4c84-9a0e-422b1ddc00be admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.731300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772491c-39d4-4b86-82b8-7c5c71312113 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.737337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.738179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.738480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.738810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.739912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.741631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.741760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddf5aa-ae8b-4a29-9dd6-c257e3704abe admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.742320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.742609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.743118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.743628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.745646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.756399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 958/1924] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:57.756399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.756399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.756399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.794102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] MFA Rules not processed for user `8eb22f7aeb0f417f962b8ba4bfc9cea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:57.795281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:57.867231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 959/1925] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:44:57.868264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.879736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a1562-3a1c-4163-b009-000ac07a8deb None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.884761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.885496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.885807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.886054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.886812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:57.889084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:57.903255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 967/1926] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:44:57 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:57.904323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.910508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 960/1927] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:57.910673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:57.912218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:57.924469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b7ca-3489-4e3a-a358-235b7a77e045 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.976556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb0d6-29f5-41b5-8ddf-80a089628749 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:57.980948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.981625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:57.981926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:57.982205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:57.998878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:57.999859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.000231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.000525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] PATH_INFO: `/v3/projects/e64a8113a009471d8be65fbc35588f69/users/ec0d24d057ce40d0b7618a8673df7b95/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.016885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64a8113a009471d8be65fbc35588f69, user_id=ec0d24d057ce40d0b7618a8673df7b95, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.019776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a4000-19ab-4ff2-acf9-038822a56cfc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.055538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.060134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.069946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 968/1928] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:58.070898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.081659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed40ff1-f93a-422d-acf1-891136bee852 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.083638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:58.088079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.088875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.089164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.089470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.090181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.092238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.125128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 961/1929] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:57 2022] PUT /identity/v3/projects/e64a8113a009471d8be65fbc35588f69/users/ec0d24d057ce40d0b7618a8673df7b95/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:58.135514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.135815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.136104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.165233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] MFA Rules not processed for user `ec0d24d057ce40d0b7618a8673df7b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:58.166608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:58.237536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 969/1930] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:44:58 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:44:58.237536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.244889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 962/1931] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:58.245682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1b513-d20f-40aa-91c1-d09d551ff521 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.245940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.311572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.312390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.312678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.312849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6276-09c7-43ce-81cc-32de9163a4ea None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.313011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] PATH_INFO: `/v3/users/752b8e20f97f42f7b185972a7c9f9f65` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.314202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] RBAC: Authorizing `identity:delete_user(user_id=752b8e20f97f42f7b185972a7c9f9f65)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.316260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.317725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.318399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.318685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.319000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.320070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.321638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.332631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 963/1932] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:44:58.333575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.336983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:58.393197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba463-fe7d-4402-a117-a984ff4ea4e0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.397064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.397731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.398020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.398306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.407464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.408908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.417922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 964/1933] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:58.418713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.432430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1574a2-c276-4062-ac65-14e1c50f85c3 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.437772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.438668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.438982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.439269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] PATH_INFO: `/v3/projects/dce9fc8202b2495fa049a488e52bb199/users/f0bbc2fd537d4d19b37f94f74cf9749b/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.457903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce9fc8202b2495fa049a488e52bb199, user_id=f0bbc2fd537d4d19b37f94f74cf9749b, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.460654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.545676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752b8e20f97f42f7b185972a7c9f9f65'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:58.546340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752b8e20f97f42f7b185972a7c9f9f65'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:58.553274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752b8e20f97f42f7b185972a7c9f9f65'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:58.581426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 965/1934] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:58 2022] PUT /identity/v3/projects/dce9fc8202b2495fa049a488e52bb199/users/f0bbc2fd537d4d19b37f94f74cf9749b/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:58.583891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.585707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:58.665749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e841d-1899-49ae-a5bb-94dbe35aac64 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.671750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.672558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.672905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.673263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.674102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.675889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.686277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 966/1935] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:44:58.690593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.702796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04c0d7-a890-4328-be4c-3d0c1f569732 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:58.707679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:58.708578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.708894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.709179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] PATH_INFO: `/v3/projects/dce9fc8202b2495fa049a488e52bb199/users/f0bbc2fd537d4d19b37f94f74cf9749b/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.711316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce9fc8202b2495fa049a488e52bb199, user_id=f0bbc2fd537d4d19b37f94f74cf9749b, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:58.715964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d79a54-1303-4bff-95bf-f8525240b8b1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:58.728040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 967/1936] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:44:58 2022] PUT /identity/v3/projects/dce9fc8202b2495fa049a488e52bb199/users/f0bbc2fd537d4d19b37f94f74cf9749b/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:58.733227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:58.742103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.742103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.742103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.769793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] MFA Rules not processed for user `f0bbc2fd537d4d19b37f94f74cf9749b`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:58.771073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:58.772676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752b8e20f97f42f7b185972a7c9f9f65'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:58.789246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b24e1893-93b9-46cc-9933-3a0e70d0734f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752b8e20f97f42f7b185972a7c9f9f65'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:58.796475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 970/1937] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:58 2022] DELETE /identity/v3/users/752b8e20f97f42f7b185972a7c9f9f65 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:44:58.801178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:58.801538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:58.802380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:58.820594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:44:58.821840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:58.851431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 968/1938] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:58.889115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 971/1939] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:44:58.902736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:58.904187 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:58.959456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22e64-37d6-448f-9839-7f3a15514713 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:59.006205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:59.006844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:59.007159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:59.007454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:59.016835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:59.018255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:59.027456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 969/1940] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:59.472542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:59.497760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:59.499247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:59.525362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:59.526909 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:44:59.544306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809ba15-fb74-4f42-ad87-2fe8ec9276a6 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:59.550074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:59.551130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:59.551382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:59.551663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] PATH_INFO: `/v3/projects/09769d07226d4816806af14f289cc49f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:59.553502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09769d07226d4816806af14f289cc49f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:59.555443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:59.567349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a67b8-704e-47b6-a3ab-73c70a1c80e3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:59.571365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:59.572025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:59.572313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:59.572593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:59.587100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:44:59.633986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:59.635510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:44:59.645525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 972/1941] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:44:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:44:59.753715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09769d07226d4816806af14f289cc49f', 'request_id': 'req-4a6faddd-2d15-46ea-a106-c57be1175a6f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:59.767551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09769d07226d4816806af14f289cc49f', 'request_id': 'req-4a6faddd-2d15-46ea-a106-c57be1175a6f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:59.895819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09769d07226d4816806af14f289cc49f', 'request_id': 'req-4a6faddd-2d15-46ea-a106-c57be1175a6f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:59.913163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] The token cache is being invalidate because project 09769d07226d4816806af14f289cc49f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:44:59.913461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:59.914161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:44:59.916230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 970/1942] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:44:59 2022] DELETE /identity/v3/projects/09769d07226d4816806af14f289cc49f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:44:59.920204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:44:59.932546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f7a5-bf76-4415-b494-75029a40db8b service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:44:59.991512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:44:59.992310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:44:59.992770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:44:59.993051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] PATH_INFO: `/v3/users/8eb22f7aeb0f417f962b8ba4bfc9cea5` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:44:59.994294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb22f7aeb0f417f962b8ba4bfc9cea5)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:44:59.998229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:00.339073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:00.352475 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb22f7aeb0f417f962b8ba4bfc9cea5'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:00.353138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb22f7aeb0f417f962b8ba4bfc9cea5'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:00.359954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb22f7aeb0f417f962b8ba4bfc9cea5'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:00.481718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb22f7aeb0f417f962b8ba4bfc9cea5'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:00.494181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb22f7aeb0f417f962b8ba4bfc9cea5'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:00.500499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 973/1943] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:44:59 2022] DELETE /identity/v3/users/8eb22f7aeb0f417f962b8ba4bfc9cea5 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:00.596206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:00.993440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:00.998831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:01.008251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faddd-2d15-46ea-a106-c57be1175a6f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.066566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.067596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.067997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.068346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] PATH_INFO: `/v3/projects/a7411eec00a3451399a785ebdfe320ae` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.069840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7411eec00a3451399a785ebdfe320ae)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.071881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.130535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.132006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:01.144546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:01.207960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37cc8d-c0c1-4c02-87ca-d68c9db4923d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.212604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bbc2fd537d4d19b37f94f74cf9749b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dce9fc8202b2495fa049a488e52bb199', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.213275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.213568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.213854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.214615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:01.214958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.216609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.227986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 974/1944] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:01.232141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.238314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7411eec00a3451399a785ebdfe320ae', 'request_id': 'req-07a97b3b-f719-4d98-8428-574ba0011734'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.241681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64ea0c-f40a-46c3-b62f-c5a0903194fd tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.244769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7411eec00a3451399a785ebdfe320ae', 'request_id': 'req-07a97b3b-f719-4d98-8428-574ba0011734'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.298458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bbc2fd537d4d19b37f94f74cf9749b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dce9fc8202b2495fa049a488e52bb199', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.299089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.299389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.299673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.300363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.302316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.363536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7411eec00a3451399a785ebdfe320ae', 'request_id': 'req-07a97b3b-f719-4d98-8428-574ba0011734'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.375407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] The token cache is being invalidate because project a7411eec00a3451399a785ebdfe320ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:01.375701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.376332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-07a97b3b-f719-4d98-8428-574ba0011734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.377874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 971/1945] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:00 2022] DELETE /identity/v3/projects/a7411eec00a3451399a785ebdfe320ae => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:01.382510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c353-0ef3-4767-b3c2-e32b788053cd None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.382805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c353-0ef3-4767-b3c2-e32b788053cd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.383128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253c353-0ef3-4767-b3c2-e32b788053cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.400886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0253c353-0ef3-4767-b3c2-e32b788053cd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:01.402095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0253c353-0ef3-4767-b3c2-e32b788053cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:01.428568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 975/1946] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:01 2022] POST /identity/v3/projects => generated 353 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:01.431305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.434294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:01.471029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 972/1947] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:01.483045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.483349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.483661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.502392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:01.503815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:01.510706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683a8f-1fa0-4acb-b0a5-d5bcd7803e43 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.517424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bbc2fd537d4d19b37f94f74cf9749b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dce9fc8202b2495fa049a488e52bb199', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.518102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.518401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.518687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.519463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:01.519813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.521394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.534959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 976/1948] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:01.567503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 973/1949] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:01.571931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.578091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.584131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b439dfb-f224-4f46-ab20-fdb85d442086 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.588824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.589371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0a5b-71eb-4394-bc82-4f89e2fec477 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.590081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.590370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.590648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.591425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:01.591765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.593424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.595206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bbc2fd537d4d19b37f94f74cf9749b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dce9fc8202b2495fa049a488e52bb199', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.596150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.596452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.596860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] PATH_INFO: `/v3/projects/284b6ab2891b4ae29f171651d759a20f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.598092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorizing `identity:delete_project(project_id=284b6ab2891b4ae29f171651d759a20f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.600036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.605704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 977/1950] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:01.609985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.619905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24ac92-9d9a-454d-baf3-18a206c0bbb6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.624854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.625509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.625803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.626083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.626777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.628693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.755077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284b6ab2891b4ae29f171651d759a20f', 'request_id': 'req-2d00f35e-fc37-454d-bd47-26a21ddb7218'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.760346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284b6ab2891b4ae29f171651d759a20f', 'request_id': 'req-2d00f35e-fc37-454d-bd47-26a21ddb7218'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.781838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:01.789326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 978/1951] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:01 2022] POST /identity/v3/projects => generated 389 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:01.790436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:01.852730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b720ea-eece-470d-a734-a9d283688e05 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:01.858248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:01.858954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:01.859268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:01.859557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:01.860364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:01.862359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:01.887051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284b6ab2891b4ae29f171651d759a20f', 'request_id': 'req-2d00f35e-fc37-454d-bd47-26a21ddb7218'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.898682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] The token cache is being invalidate because project 284b6ab2891b4ae29f171651d759a20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:01.899089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.899735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:01.901290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 974/1952] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:01 2022] DELETE /identity/v3/projects/284b6ab2891b4ae29f171651d759a20f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:02.029729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 979/1953] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:01 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:02.033789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.035216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:02.099311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00f35e-fc37-454d-bd47-26a21ddb7218 tempest-VolumeQuotaClassesTest-190608045 tempest-VolumeQuotaClassesTest-190608045-project-admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.104038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.104736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.105023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.105304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.106062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.107688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.117927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 975/1954] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:02.122636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.132815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c20e-0eba-48ac-b26a-90654d41b165 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.138506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.139198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.139491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.139772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.140511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.142121 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973341cd-777c-4887-b771-95447e1d474b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.153492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 980/1955] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:02.156717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.167086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975c65-2623-47c0-8464-cc5112d6b5eb admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.172131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.173021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.173310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.173587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] PATH_INFO: `/v3/projects/0b57000f1ac942e5b5275b5a3a247090/users/2ed7683dcf8f4ae280613885da576de1/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.190988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b57000f1ac942e5b5275b5a3a247090, user_id=2ed7683dcf8f4ae280613885da576de1, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.193823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.293592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 976/1956] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:02 2022] PUT /identity/v3/projects/0b57000f1ac942e5b5275b5a3a247090/users/2ed7683dcf8f4ae280613885da576de1/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:02.304902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.305202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.305509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.332400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] MFA Rules not processed for user `2ed7683dcf8f4ae280613885da576de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:02.333630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:02.395877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 981/1957] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:02.400353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.462401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e411-779a-4d5b-a831-1c6fb1858fe4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.467547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.468260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.468562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.468845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.469553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.471522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.558139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:02.614485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 977/1958] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:02 2022] POST /identity/v3/projects => generated 389 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:02.618315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.632448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae7fe-12a8-4922-8fea-d553c4c90fb3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.691584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.692278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.692564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.692845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.693599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.695845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.864188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.866576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:02.880449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244df00-4bf7-47e6-ac6c-93d3e5f7fa40 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.885352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.886835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.887166 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.887443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] PATH_INFO: `/v3/users/6cf8849f2b4d408695d73a6b55bc5632` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.888709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf8849f2b4d408695d73a6b55bc5632)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.890670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.904638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 982/1959] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:02 2022] POST /identity/v3/users => generated 301 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:02.920211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.930099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57243-be5d-4417-9c93-c2c55c9b9a66 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.937671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.937671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.937671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.937671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.938066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.939315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:02.955096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 983/1960] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:02.958255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:02.968148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92bad-99b5-4d51-8f18-f36b74d9114b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:02.972911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:02.973802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:02.974087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:02.974368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] PATH_INFO: `/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04/users/7cbb684c0cfe4d6b9316019f4d8a4313/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:02.989735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2396abdf27542cb8cf12ba66a2e1c04, user_id=7cbb684c0cfe4d6b9316019f4d8a4313, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:02.993034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.089472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 984/1961] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:02 2022] PUT /identity/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04/users/7cbb684c0cfe4d6b9316019f4d8a4313/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:03.093612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.126091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf8849f2b4d408695d73a6b55bc5632'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.127020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf8849f2b4d408695d73a6b55bc5632'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.134513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf8849f2b4d408695d73a6b55bc5632'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.158675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3cc05-2589-4149-8efc-66cc32162d35 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.163646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.164327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.164622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.164907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.165649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.167808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.181565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 985/1962] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:03.186407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.188203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.201353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64021e94-9e93-4c90-8103-344f9523aef5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.206098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.206974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.207267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.207548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] PATH_INFO: `/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04/users/7cbb684c0cfe4d6b9316019f4d8a4313/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.209031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf8849f2b4d408695d73a6b55bc5632'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.209616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2396abdf27542cb8cf12ba66a2e1c04, user_id=7cbb684c0cfe4d6b9316019f4d8a4313, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.212367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369de00-d62b-4ab6-8d28-5e0962b99260 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.222209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-aee09214-73d0-4046-9fa9-74b04a3ecc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf8849f2b4d408695d73a6b55bc5632'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.222799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 986/1963] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:03 2022] PUT /identity/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04/users/7cbb684c0cfe4d6b9316019f4d8a4313/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:03.232287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 978/1964] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:02 2022] DELETE /identity/v3/users/6cf8849f2b4d408695d73a6b55bc5632 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:03.233504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.233802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.234087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.236642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.237155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.237762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.256091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:03.257308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.260720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] MFA Rules not processed for user `7cbb684c0cfe4d6b9316019f4d8a4313`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:03.261903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.315990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 979/1965] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:03.325871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.327428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.332917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 987/1966] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:03.344773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.346135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.385774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b069d-94d8-4253-91b9-ebe407670b01 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.389517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.390159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.390444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.390738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.399836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.401272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.404207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4716d-13f1-4de3-b28d-3b7ae069ccec None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.408170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.408802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.409093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.409373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.411430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 980/1967] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:03.420159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.421621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.432019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 988/1968] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:03.434017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:03.470308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:03.560848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.571272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438147-b125-4179-9d98-a12669ae4e3b service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.576045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.576912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.577229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.577512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] PATH_INFO: `/v3/users/2ed7683dcf8f4ae280613885da576de1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.578762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed7683dcf8f4ae280613885da576de1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.580717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.809231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed7683dcf8f4ae280613885da576de1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.810040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed7683dcf8f4ae280613885da576de1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.816884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed7683dcf8f4ae280613885da576de1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.837559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:03.843862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:03.866440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0b672-eff0-4f44-9900-49f906482bde service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:03.879022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:03.879022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.879022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.879022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] PATH_INFO: `/v3/projects/d844855fe5c44ec19d4102f0b6d08ffd` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.879707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d844855fe5c44ec19d4102f0b6d08ffd)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:03.879707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:03.892401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed7683dcf8f4ae280613885da576de1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.906430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f651cac7-4dd1-4997-91d4-9e59225d54d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed7683dcf8f4ae280613885da576de1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:03.912644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 981/1969] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:03 2022] DELETE /identity/v3/users/2ed7683dcf8f4ae280613885da576de1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:03.917052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:03.917373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:03.917654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:03.941449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:03.941449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:04.002044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 982/1970] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:04.006404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd844855fe5c44ec19d4102f0b6d08ffd', 'request_id': 'req-506f2571-f25e-43ab-a1bd-6bc247ae13e6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.012077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:04.012554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd844855fe5c44ec19d4102f0b6d08ffd', 'request_id': 'req-506f2571-f25e-43ab-a1bd-6bc247ae13e6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.013437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:04.070713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc1e7-5c69-4e95-bc3b-eea22bbdd278 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:04.110132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd844855fe5c44ec19d4102f0b6d08ffd', 'request_id': 'req-506f2571-f25e-43ab-a1bd-6bc247ae13e6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.121087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:04.121838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:04.122194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:04.122591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:04.125823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] The token cache is being invalidate because project d844855fe5c44ec19d4102f0b6d08ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:04.126159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.127300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.128975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 989/1971] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:03 2022] DELETE /identity/v3/projects/d844855fe5c44ec19d4102f0b6d08ffd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:04.132870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:04.134311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:04.196659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:04.199822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:04.217405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 983/1972] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:04.224361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2571-f25e-43ab-a1bd-6bc247ae13e6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:04.229400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:04.230324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:04.232857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:04.232857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] PATH_INFO: `/v3/users/f0bbc2fd537d4d19b37f94f74cf9749b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:04.232857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0bbc2fd537d4d19b37f94f74cf9749b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:04.234264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:04.462336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0bbc2fd537d4d19b37f94f74cf9749b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.463237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0bbc2fd537d4d19b37f94f74cf9749b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.470307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0bbc2fd537d4d19b37f94f74cf9749b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.510166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:04.532843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0bbc2fd537d4d19b37f94f74cf9749b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.547320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0bbc2fd537d4d19b37f94f74cf9749b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.553516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 990/1973] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:04 2022] DELETE /identity/v3/users/f0bbc2fd537d4d19b37f94f74cf9749b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:04.672314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:04.673926 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:04.686865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75fd98-7043-492d-a46f-3890ce7f17b0 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:04.748171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:04.749096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:04.749378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:04.749654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] PATH_INFO: `/v3/projects/0b57000f1ac942e5b5275b5a3a247090` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:04.750857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b57000f1ac942e5b5275b5a3a247090)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:04.752790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:04.797807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:04.887589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b57000f1ac942e5b5275b5a3a247090', 'request_id': 'req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.894419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b57000f1ac942e5b5275b5a3a247090', 'request_id': 'req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.974646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b57000f1ac942e5b5275b5a3a247090', 'request_id': 'req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.988484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] The token cache is being invalidate because project 0b57000f1ac942e5b5275b5a3a247090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:04.988861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.990064 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:04.991995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 984/1974] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:04 2022] DELETE /identity/v3/projects/0b57000f1ac942e5b5275b5a3a247090 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:04.995809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:04.997173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:05.010425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa7753-b350-4c93-90a1-a24ef2c6b661 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.074017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.075291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.075580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.075857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] PATH_INFO: `/v3/users/7cbb684c0cfe4d6b9316019f4d8a4313` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.077071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbb684c0cfe4d6b9316019f4d8a4313)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:05.084632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:05.113248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:05.114721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:05.132488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e2d4-6e06-4f86-b07b-71e431be5b36 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.137799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.138690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.139080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.139394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] PATH_INFO: `/v3/projects/dce9fc8202b2495fa049a488e52bb199` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.140632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce9fc8202b2495fa049a488e52bb199)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:05.142556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:05.296848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbb684c0cfe4d6b9316019f4d8a4313'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.297033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce9fc8202b2495fa049a488e52bb199', 'request_id': 'req-5504cf08-c2d9-4a72-b469-53bc18d24f79'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.297860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbb684c0cfe4d6b9316019f4d8a4313'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.303903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce9fc8202b2495fa049a488e52bb199', 'request_id': 'req-5504cf08-c2d9-4a72-b469-53bc18d24f79'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.305776 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbb684c0cfe4d6b9316019f4d8a4313'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.404065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce9fc8202b2495fa049a488e52bb199', 'request_id': 'req-5504cf08-c2d9-4a72-b469-53bc18d24f79'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.415489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] The token cache is being invalidate because project dce9fc8202b2495fa049a488e52bb199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:05.415784 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.416434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5504cf08-c2d9-4a72-b469-53bc18d24f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.418003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 985/1975] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:05 2022] DELETE /identity/v3/projects/dce9fc8202b2495fa049a488e52bb199 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:05.422701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.423032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.423318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.440998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:05.442239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:05.463747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbb684c0cfe4d6b9316019f4d8a4313'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.477722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-57fbf120-dad6-4801-a46d-4dc9438271d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbb684c0cfe4d6b9316019f4d8a4313'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:05.483909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 991/1976] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:04 2022] DELETE /identity/v3/users/7cbb684c0cfe4d6b9316019f4d8a4313 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:05.508012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 986/1977] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:05.516016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.516456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.516809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.535570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:05.536858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:05.601613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 992/1978] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:05.605872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:05.615477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7cf86-c01b-43e5-9f8c-d745e0cfb58c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.620060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.621236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.621525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.621807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.622535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:05.622849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:05.624527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:05.635782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 987/1979] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:05.640590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:05.649735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42f81e-0054-47a6-88e6-b9a25e0180ca None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.654342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.655031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.655324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.655605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.656307 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:05.658201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:05.794464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 993/1980] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:05 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:05.795727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:05.805680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90348ade-85ab-4c99-aa3d-3bb98df4fe9b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.862127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.862795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:05.863125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:05.863405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:05.864105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:05.866079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:05.976589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:05.978336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:05.992738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3b96-8156-4785-9518-1bffca19c4ed admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:05.997868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:05.999802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.000095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.000419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] PATH_INFO: `/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.001574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2396abdf27542cb8cf12ba66a2e1c04)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.003566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.012527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 988/1981] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:05 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:06.015401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.025144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a5157-ad1e-4863-8979-d8cf0d711703 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.030453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.031149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.031442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.031724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.032475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.034063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.047070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 989/1982] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:06.048344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.059716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62344a6-c700-44b7-b917-f7e9fb49f6f3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.064422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.065108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.065394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.065673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.066408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.068014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.079929 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 990/1983] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:06.090268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.100088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bdf3e-bf4c-4906-bd2a-7bbfd9407897 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.104690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.105558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.105846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.106202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] PATH_INFO: `/v3/projects/6e908e9e04264519bc2ee336d504bb43/users/8e31274f49be48a3ad24f956e96bc8e0/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.112432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:06.120419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e908e9e04264519bc2ee336d504bb43, user_id=8e31274f49be48a3ad24f956e96bc8e0, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.123047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ce086-0ea7-491b-aab9-36353cd86935 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.216336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 991/1984] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:06 2022] PUT /identity/v3/projects/6e908e9e04264519bc2ee336d504bb43/users/8e31274f49be48a3ad24f956e96bc8e0/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:06.225490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2396abdf27542cb8cf12ba66a2e1c04', 'request_id': 'req-50e9b34d-27ed-4c41-aed4-4e47dd86a039'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:06.228761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.229106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.229398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.231762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2396abdf27542cb8cf12ba66a2e1c04', 'request_id': 'req-50e9b34d-27ed-4c41-aed4-4e47dd86a039'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:06.257551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] MFA Rules not processed for user `8e31274f49be48a3ad24f956e96bc8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:06.259078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:06.324023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2396abdf27542cb8cf12ba66a2e1c04', 'request_id': 'req-50e9b34d-27ed-4c41-aed4-4e47dd86a039'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:06.324792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 992/1985] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:06.329348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.336645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] The token cache is being invalidate because project c2396abdf27542cb8cf12ba66a2e1c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:06.336936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:06.337578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-50e9b34d-27ed-4c41-aed4-4e47dd86a039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:06.339738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 994/1986] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:05 2022] DELETE /identity/v3/projects/c2396abdf27542cb8cf12ba66a2e1c04 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:06.344338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b3f91-9cb7-4dfd-91d8-895174e1c138 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.344695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b3f91-9cb7-4dfd-91d8-895174e1c138 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.345057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b3f91-9cb7-4dfd-91d8-895174e1c138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.362840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-343b3f91-9cb7-4dfd-91d8-895174e1c138 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:06.364223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-343b3f91-9cb7-4dfd-91d8-895174e1c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:06.391895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69917741-6dc4-49d9-9e3c-cf3f5ca1dbee None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.396667 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.397360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.397648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.397928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.398624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.400576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.427643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 995/1987] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:06.436051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.436359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.436642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.455687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:06.456912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:06.465858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 996/1988] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:06.469115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 993/1989] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:06.472028 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.472819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.475298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:06.481572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84440ee2-d958-438a-9f8f-40c665b2b352 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.545362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.546143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.546511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.547031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.547850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:06.548175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.549673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5e867-ddb4-4386-85ec-09412909fc17 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.549797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.559002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.559696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.560013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.560303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.561013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.563030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.563333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 997/1990] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:06.568019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.578063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57025-d92e-41b1-a4cb-d817a42bd9b5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.582847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.583551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.584188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.584188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.584840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.586750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.655194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 994/1991] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:06.658875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.668710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e355-d4d4-4358-a4c9-fc59fb14809a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.673353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.673966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.674255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.674539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.675324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.676899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.687068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 995/1992] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:06.688648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 998/1993] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:06.690524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.692429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.700389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d64f6-e93f-4a8f-a90d-1518cf771fd3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.754098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e525-587c-4396-a8a6-2d94ab7e4a8e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.755164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.755839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.756126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.756402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.757135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.758727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.758833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.760738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.761559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.761841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.762545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.764477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.770067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 996/1994] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:06.774100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.783707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cf432-25b3-40f2-a9f5-facd9ad0d5b1 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.788488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.789358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.789649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.789932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] PATH_INFO: `/v3/projects/83ee9d4b8c3c4d20a715419563eb70a9/users/bdbad58c2f50462c99a46e5dc4763117/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.805108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ee9d4b8c3c4d20a715419563eb70a9, user_id=bdbad58c2f50462c99a46e5dc4763117, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.807888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb9b24-0f38-4175-8fb8-c3ad15d4cf00 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.861328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 999/1995] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:06.866821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.877393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8794-3b76-40fa-8aa9-be76b70d4052 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.882074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.882734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.883061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.883347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.884078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.885739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.895623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1000/1996] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:06.899459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.924087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53ba56-66b7-4f66-8ed5-f565895e2183 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.924087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:06.924087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.924087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.924529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.924529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:06.924529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:06.930231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1001/1997] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:06.934865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:06.944047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cfcaf-b998-47d1-adda-a86f2e34f739 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:06.950167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 997/1998] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:06 2022] PUT /identity/v3/projects/83ee9d4b8c3c4d20a715419563eb70a9/users/bdbad58c2f50462c99a46e5dc4763117/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:06.960093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:06.960384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:06.960663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:06.987208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] MFA Rules not processed for user `bdbad58c2f50462c99a46e5dc4763117`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:06.988494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:07.000163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.001066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.001343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.001656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] PATH_INFO: `/v3/projects/4951f4c9f1d44b5fa77543f32ce90808/users/5c10f5ac85ad42d6b67361c6bbe040ae/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.016031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4951f4c9f1d44b5fa77543f32ce90808, user_id=5c10f5ac85ad42d6b67361c6bbe040ae, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.018667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f44bd-2a4c-4f76-86be-ee3fde9220cd admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.052673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 998/1999] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:07.057124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.065453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:07.071440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d3f7e-b7f6-4e75-8747-323a13cc8113 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.076095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.076749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.077040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.077321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.078022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.079960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.132079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1002/2000] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:06 2022] PUT /identity/v3/projects/4951f4c9f1d44b5fa77543f32ce90808/users/5c10f5ac85ad42d6b67361c6bbe040ae/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:07.137508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.137806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.138082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.169754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] MFA Rules not processed for user `5c10f5ac85ad42d6b67361c6bbe040ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:07.171110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:07.224993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 999/2001] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:07 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:07.231534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.250979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1003/2002] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:07.262720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.266337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:07.305255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81c42-57e2-444a-ab94-f3b4238e5267 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.310765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.311468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.311757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.312069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.312770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.314752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.324686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9849-6a6f-4998-a0fa-ae3f2daf010c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.331371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:07.377330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.378021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.378313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.378597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.433864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1000/2003] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:07 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:07.438075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.439196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.440772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.449026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7ddee-1d87-4def-92ec-4861c3616b4f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.452017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1004/2004] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:07.454150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.454824 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.455157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.455446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.456180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.458371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.474097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1001/2005] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:07.484398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.496012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebc853-1eca-4b52-9de5-42ec6c899127 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.500723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.501642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.501936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.502229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] PATH_INFO: `/v3/projects/c19e40dae4e2498789fac9ce408982d1/users/ea70d49a0a9a4a0fba98d82bc9b59d3c/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.518517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19e40dae4e2498789fac9ce408982d1, user_id=ea70d49a0a9a4a0fba98d82bc9b59d3c, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.521260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.658506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1005/2006] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:07 2022] PUT /identity/v3/projects/c19e40dae4e2498789fac9ce408982d1/users/ea70d49a0a9a4a0fba98d82bc9b59d3c/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:07.665272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.676412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8237-e50a-432c-bc8f-6ef02807727f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.749395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.750142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.750431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.750702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.751464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.753093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997ec14-ae44-4195-bf77-be8e577b22c9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.764197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1002/2007] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:07.768457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:07.778948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017702-1da4-4764-a1e8-743ddec0d521 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:07.783709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:07.784593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.784879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.785160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] PATH_INFO: `/v3/projects/c19e40dae4e2498789fac9ce408982d1/users/ea70d49a0a9a4a0fba98d82bc9b59d3c/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.787263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19e40dae4e2498789fac9ce408982d1, user_id=ea70d49a0a9a4a0fba98d82bc9b59d3c, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:07.790019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:07.809810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1006/2008] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:07 2022] PUT /identity/v3/projects/c19e40dae4e2498789fac9ce408982d1/users/ea70d49a0a9a4a0fba98d82bc9b59d3c/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:07.820429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:07.820722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:07.821008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:07.848270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] MFA Rules not processed for user `ea70d49a0a9a4a0fba98d82bc9b59d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:07.849542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:07.928053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1003/2009] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:08.083632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:08.086838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:08.147772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a87ff-f8f4-4c73-b2ed-ef0d6e4a67a8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:08.151429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:08.152138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:08.152427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:08.155224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:08.161982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:08.163471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:08.173733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1007/2010] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:08.277119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:08.352831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:08.379567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:08.393339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f63b-75a4-4e38-82fd-deefd93effa7 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:08.398155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:08.399610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:08.401005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:08.401005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:08.403672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:08.497250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:08.497250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:08.509831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1004/2011] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:09.791998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:10.080916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:11.055653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:11.220637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:11.658234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:11.723767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:12.083487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:12.266359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:12.543780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:12.629621 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:16.149710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:16.163636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:16.224730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205d10f-27ef-44dc-9864-e253b2e001d7 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:16.229662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:16.230336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a710f-9475-4f6a-a37b-2579503a99dd service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:16.230550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.230841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.231625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] PATH_INFO: `/v3/users/5c10f5ac85ad42d6b67361c6bbe040ae` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.232575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c10f5ac85ad42d6b67361c6bbe040ae)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:16.234504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:16.235382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:16.236272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.236685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.236970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] PATH_INFO: `/v3/users/8e31274f49be48a3ad24f956e96bc8e0` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.238279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e31274f49be48a3ad24f956e96bc8e0)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:16.240314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:16.469974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e31274f49be48a3ad24f956e96bc8e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.470749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e31274f49be48a3ad24f956e96bc8e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.473709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c10f5ac85ad42d6b67361c6bbe040ae'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.474433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c10f5ac85ad42d6b67361c6bbe040ae'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.479269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e31274f49be48a3ad24f956e96bc8e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.485131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c10f5ac85ad42d6b67361c6bbe040ae'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.617850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e31274f49be48a3ad24f956e96bc8e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.631525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6c06ebf-25c6-471a-8c1b-22c0f6dfedbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e31274f49be48a3ad24f956e96bc8e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.637531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1005/2012] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:16 2022] DELETE /identity/v3/users/8e31274f49be48a3ad24f956e96bc8e0 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:16.641998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.642291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.642572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.660557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:16.672537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:16.712223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c10f5ac85ad42d6b67361c6bbe040ae'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.725235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:16.734111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-846c075a-6284-42e1-b12b-20096b55e3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c10f5ac85ad42d6b67361c6bbe040ae'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:16.741767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1008/2013] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:16 2022] DELETE /identity/v3/users/5c10f5ac85ad42d6b67361c6bbe040ae => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:16.747811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.748205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.748617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.750960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1006/2014] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:16.767817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:16.767973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:16.767973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:16.773722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:16.775129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:16.828053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ae84b-9787-4b0b-9aef-54f6cfc82d12 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:16.837208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1009/2015] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:16.847496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:16.883203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:16.884569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.884863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.885146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.896322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:16.897782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:16.905411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e525cf4-0f76-4278-8ee8-bd3654c31980 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:16.908644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1007/2016] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:16.909183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:16.909878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:16.910447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:16.910739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:16.924116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:16.930436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:16.938465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1010/2017] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:17.584517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:17.586614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:17.612146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715456a-4c4d-4789-9d59-0bdda4151ca3 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:17.618812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:17.619800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:17.620811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:17.621140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] PATH_INFO: `/v3/projects/6e908e9e04264519bc2ee336d504bb43` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:17.623464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e908e9e04264519bc2ee336d504bb43)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:17.626242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:17.914916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:17.917923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:17.934608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b371-b5fa-4de3-b501-f52e53b5cad8 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:17.940339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e908e9e04264519bc2ee336d504bb43', 'request_id': 'req-ed797585-20b7-4b83-91e0-242f4154d0f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:17.951440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e908e9e04264519bc2ee336d504bb43', 'request_id': 'req-ed797585-20b7-4b83-91e0-242f4154d0f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.002483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:18.003416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.003704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.003986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] PATH_INFO: `/v3/projects/4951f4c9f1d44b5fa77543f32ce90808` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.005789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4951f4c9f1d44b5fa77543f32ce90808)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:18.007755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:18.060213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e908e9e04264519bc2ee336d504bb43', 'request_id': 'req-ed797585-20b7-4b83-91e0-242f4154d0f3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.079826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] The token cache is being invalidate because project 6e908e9e04264519bc2ee336d504bb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:18.080123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.080786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.082619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1008/2018] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:17 2022] DELETE /identity/v3/projects/6e908e9e04264519bc2ee336d504bb43 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:18.086458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:18.087932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:18.101501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed797585-20b7-4b83-91e0-242f4154d0f3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:18.107190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:18.108125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.108410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.108685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] PATH_INFO: `/v3/users/bdbad58c2f50462c99a46e5dc4763117` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.110029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbad58c2f50462c99a46e5dc4763117)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:18.112081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:18.140772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:18.165246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:18.167336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4951f4c9f1d44b5fa77543f32ce90808', 'request_id': 'req-61a96d8f-9262-4d93-89e8-96a5e4648009'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.174103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4951f4c9f1d44b5fa77543f32ce90808', 'request_id': 'req-61a96d8f-9262-4d93-89e8-96a5e4648009'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.328843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4951f4c9f1d44b5fa77543f32ce90808', 'request_id': 'req-61a96d8f-9262-4d93-89e8-96a5e4648009'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.374376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] The token cache is being invalidate because project 4951f4c9f1d44b5fa77543f32ce90808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:18.375296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.375595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61a96d8f-9262-4d93-89e8-96a5e4648009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.377262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1011/2019] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:17 2022] DELETE /identity/v3/projects/4951f4c9f1d44b5fa77543f32ce90808 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:18.382697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbd607-4c3c-4750-89ee-8253d3ee77a7 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.382697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbd607-4c3c-4750-89ee-8253d3ee77a7 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.382853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbd607-4c3c-4750-89ee-8253d3ee77a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.410569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-98fbd607-4c3c-4750-89ee-8253d3ee77a7 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:18.413771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-98fbd607-4c3c-4750-89ee-8253d3ee77a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:18.483891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbad58c2f50462c99a46e5dc4763117'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.486269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbad58c2f50462c99a46e5dc4763117'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.504664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbad58c2f50462c99a46e5dc4763117'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.522496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1012/2020] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:18.531089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.531447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.531708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.551989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:18.553323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:18.562698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1013/2021] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:18.566721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:18.577969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a5a1c-12f0-455b-86ac-c0f69f91b9a5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:18.582953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:18.584114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.584582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.584912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.585885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:18.586202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:18.587881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:18.606684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1014/2022] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:18.607737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:18.617985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504551c-e443-4df2-a076-f6e8141fd4fa admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:18.623342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:18.624134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:18.624452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:18.624731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:18.625429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:18.627395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:18.640979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbad58c2f50462c99a46e5dc4763117'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.654012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbad58c2f50462c99a46e5dc4763117'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:18.660567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1009/2023] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:18 2022] DELETE /identity/v3/users/bdbad58c2f50462c99a46e5dc4763117 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:18.882651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1015/2024] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:18 2022] POST /identity/v3/projects => generated 375 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:18.887083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:18.954319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed672-9385-4348-8826-7816c4e5cf57 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.011662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.012370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.012785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.012955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.013647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.015616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.203773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1010/2025] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:18 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:19.208218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:19.233825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b5b4a-a797-475c-8d8a-9650f439880f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.239827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.240707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.241093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.241643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.242511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.244456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.257619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1016/2026] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:19.263819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:19.275700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed9dd2-f65d-4216-8c67-aa4ffb4c7394 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.281838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.282699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.283103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.283389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.284376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.286513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.298650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1011/2027] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:19.302986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:19.316881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33ec7-6005-49db-8f09-5c8d92952f0e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.326673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.328428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.329064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.329527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] PATH_INFO: `/v3/projects/7685b2d9079d4073b4c8013cfe9a7cd3/users/d218157c531a4e4c9900649d175ed249/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.360384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7685b2d9079d4073b4c8013cfe9a7cd3, user_id=d218157c531a4e4c9900649d175ed249, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.363165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddfb69-fd14-430a-9d34-54c7dbd9c1e2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.478629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:19.481242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:19.489709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1017/2028] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:19 2022] PUT /identity/v3/projects/7685b2d9079d4073b4c8013cfe9a7cd3/users/d218157c531a4e4c9900649d175ed249/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:19.491301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.491600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.491889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.504509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63815c4a-b7b5-42a2-9274-68e76e6e148a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.545584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] MFA Rules not processed for user `d218157c531a4e4c9900649d175ed249`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:19.547338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:19.601328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.603031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.603031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.603031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] PATH_INFO: `/v3/projects/83ee9d4b8c3c4d20a715419563eb70a9` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.604224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ee9d4b8c3c4d20a715419563eb70a9)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.606818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.628646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1018/2029] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:19.636052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:19.647057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789d55d-8e7b-4212-b2db-da6bc29e96d6 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:19.652748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:19.653170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:19.653461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:19.653750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:19.654454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:19.656409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:19.861308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ee9d4b8c3c4d20a715419563eb70a9', 'request_id': 'req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:19.866718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ee9d4b8c3c4d20a715419563eb70a9', 'request_id': 'req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:19.956727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1019/2030] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:19 2022] POST /identity/v3/projects => generated 375 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:19.960491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.020885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b651e28-c641-471a-9d78-d9bfdbfd88b0 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.025401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.026057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.026897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.027204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.027898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.029811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.140210 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ee9d4b8c3c4d20a715419563eb70a9', 'request_id': 'req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.152522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] The token cache is being invalidate because project 83ee9d4b8c3c4d20a715419563eb70a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:20.152809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.153436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.155002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1012/2031] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:19 2022] DELETE /identity/v3/projects/83ee9d4b8c3c4d20a715419563eb70a9 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:20.158770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.160107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:20.199045 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1020/2032] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:19 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:20.204719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.210737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:20.228452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1409f9-628a-49b9-8fe5-3e0c2d16bdf2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.228883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f533-a1c1-4714-b89b-506830185050 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.234823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.235896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.236104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.236372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] PATH_INFO: `/v3/users/ea70d49a0a9a4a0fba98d82bc9b59d3c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.238245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea70d49a0a9a4a0fba98d82bc9b59d3c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.240153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.240321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.241599 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.242040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.243133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.243943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.246088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.259055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1021/2033] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:20.263753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.279110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd2f0-3ca3-4a6d-a909-24b41802c3fb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.280701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.281391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.281708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.281981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.282717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.284373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.297433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1022/2034] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:20.301608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.312302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a66d-b2cd-425a-b746-0bda7e66fdaa admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.317319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.318225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.318513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.318795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] PATH_INFO: `/v3/projects/af3d25e97bdf43d7bcf595728698e115/users/e9ebc6c9b6e54f5a89b3709d374c6880/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.335119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3d25e97bdf43d7bcf595728698e115, user_id=e9ebc6c9b6e54f5a89b3709d374c6880, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.338040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3dad6-40e9-4a00-9cbd-a5c4f1177ac6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.483218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:20.534843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1023/2035] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:20 2022] PUT /identity/v3/projects/af3d25e97bdf43d7bcf595728698e115/users/e9ebc6c9b6e54f5a89b3709d374c6880/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:20.544692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.544989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.545273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.571884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] MFA Rules not processed for user `e9ebc6c9b6e54f5a89b3709d374c6880`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:20.577098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:20.635936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea70d49a0a9a4a0fba98d82bc9b59d3c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.637167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea70d49a0a9a4a0fba98d82bc9b59d3c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.650850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1024/2036] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:20.650989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea70d49a0a9a4a0fba98d82bc9b59d3c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.651087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.764095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b98c84-b04e-4e2f-9244-b95e804a2667 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:20.770677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:20.771400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:20.772209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:20.772209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:20.773229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:20.775246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:20.837023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea70d49a0a9a4a0fba98d82bc9b59d3c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.865026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea70d49a0a9a4a0fba98d82bc9b59d3c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:20.876199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1013/2037] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:20 2022] DELETE /identity/v3/users/ea70d49a0a9a4a0fba98d82bc9b59d3c => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:20.926223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1025/2038] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:20 2022] POST /identity/v3/projects => generated 377 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:20.929960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:20.931785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:21.007554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0fdc-f3d1-4148-9615-7a8bf75d93cb admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.070321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.071100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.071457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.071801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.072597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.074752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.321878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1014/2039] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:20 2022] POST /identity/v3/users => generated 295 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:21.322080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.335383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cff2c-bebe-4323-95f6-8bb4707dd11c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.341200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.341614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.342019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.342367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.343387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.345213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.362288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1026/2040] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:21.367016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.379949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2a691-c004-4bad-a0c9-a6ea0a8ef08b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.384841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.385849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.396318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.396318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] PATH_INFO: `/v3/projects/f47539aa5ebb472c89aaf74eb5106d07/users/1a1d6a8d1ac24b1a8ed2887b37a90d57/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.403424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47539aa5ebb472c89aaf74eb5106d07, user_id=1a1d6a8d1ac24b1a8ed2887b37a90d57, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.406480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.577370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1015/2041] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:21 2022] PUT /identity/v3/projects/f47539aa5ebb472c89aaf74eb5106d07/users/1a1d6a8d1ac24b1a8ed2887b37a90d57/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:21.579261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.597520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce6af-1da3-4b8d-b445-bbbeef38dba0 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.624106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.628928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:21.694368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.696111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.696403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.696686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.697427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.699098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27204f8a-0987-4673-972e-d3138c57d1ef admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.710237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1027/2042] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:21.711977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b911d-594b-4ffd-b947-3386bd82ba88 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.712139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f63b-4f39-4062-b489-fe1b3c9361d5 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.712439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f63b-4f39-4062-b489-fe1b3c9361d5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.712794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f63b-4f39-4062-b489-fe1b3c9361d5 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.714411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1028/2043] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:45:21.715131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434f63b-4f39-4062-b489-fe1b3c9361d5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.718167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.719281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.719652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.719941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] PATH_INFO: `/v3/projects/c19e40dae4e2498789fac9ce408982d1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.721139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19e40dae4e2498789fac9ce408982d1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.723230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.729959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434f63b-4f39-4062-b489-fe1b3c9361d5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:21.735600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:21.736500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.736798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.737095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] PATH_INFO: `/v3/projects/f47539aa5ebb472c89aaf74eb5106d07/users/1a1d6a8d1ac24b1a8ed2887b37a90d57/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.739261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47539aa5ebb472c89aaf74eb5106d07, user_id=1a1d6a8d1ac24b1a8ed2887b37a90d57, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:21.742096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e275-18f5-4c6e-88a0-4a728e79ff45 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:21.754892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1029/2044] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:21 2022] PUT /identity/v3/projects/f47539aa5ebb472c89aaf74eb5106d07/users/1a1d6a8d1ac24b1a8ed2887b37a90d57/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:21.756202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00e29-a509-4c0a-9d0b-d07b6529f69b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.756681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00e29-a509-4c0a-9d0b-d07b6529f69b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.756861 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00e29-a509-4c0a-9d0b-d07b6529f69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.777446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-04e00e29-a509-4c0a-9d0b-d07b6529f69b None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:21.778647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-04e00e29-a509-4c0a-9d0b-d07b6529f69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:21.797355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:21.842495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1030/2045] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:45:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:21.843757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:21.844060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:21.844417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:21.874292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] MFA Rules not processed for user `1a1d6a8d1ac24b1a8ed2887b37a90d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:21.875801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:21.968793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1031/2046] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:21.969647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:21.971439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:21.977589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19e40dae4e2498789fac9ce408982d1', 'request_id': 'req-9da5063a-2327-41b9-9894-f0fdac033f7e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:21.984345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19e40dae4e2498789fac9ce408982d1', 'request_id': 'req-9da5063a-2327-41b9-9894-f0fdac033f7e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:21.993917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a9519-5ef1-4180-831f-d6a6b807cf29 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.056314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.056972 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.057773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.058054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.066490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.068104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:22.078777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1032/2047] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:22.079622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:22.080928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.152526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb124-b5cf-4b00-9167-790135c6ac49 service placement] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.172595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.172595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.172595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.172595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] PATH_INFO: `/v3/users/d9320ca5b2a9499a86d5614b55eddede` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.173626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9320ca5b2a9499a86d5614b55eddede)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.173626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:22.214369 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19e40dae4e2498789fac9ce408982d1', 'request_id': 'req-9da5063a-2327-41b9-9894-f0fdac033f7e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.225894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] The token cache is being invalidate because project c19e40dae4e2498789fac9ce408982d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:22.226185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.226834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.228594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1016/2048] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:21 2022] DELETE /identity/v3/projects/c19e40dae4e2498789fac9ce408982d1 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:22.229999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:22.231362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.302735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5063a-2327-41b9-9894-f0fdac033f7e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.307221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.308447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.308730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.309003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.310791 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.368875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.370938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1036f1f-ed9d-4fab-859c-34a4dfe479c4 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:22.380020 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1017/2049] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:22.390300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f1f-059a-4463-af91-17081de41c1b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.390300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f1f-059a-4463-af91-17081de41c1b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.390300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f1f-059a-4463-af91-17081de41c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.400629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d6fa0f1f-059a-4463-af91-17081de41c1b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:22.401863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d6fa0f1f-059a-4463-af91-17081de41c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.412513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1018/2050] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:22.429006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.429382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.429773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.450098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:22.451298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.460834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1019/2051] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:22.465318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:22.477922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02752827-b3fa-4dab-8a7f-4f1f3dc67f56 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.489113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.489787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.490074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.490361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.491178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:22.491440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.493093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:22.519671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1020/2052] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:22.520870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:22.548225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9320ca5b2a9499a86d5614b55eddede'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.549208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9320ca5b2a9499a86d5614b55eddede'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.550264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a4a5-0ceb-45d6-8878-702688d1f2f9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.557985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9320ca5b2a9499a86d5614b55eddede'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.570181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.570848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.571161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.571901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.574283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.576510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:22.708945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9320ca5b2a9499a86d5614b55eddede'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.723483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9320ca5b2a9499a86d5614b55eddede'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:22.730454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1033/2053] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:22 2022] DELETE /identity/v3/users/d9320ca5b2a9499a86d5614b55eddede => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:22.784331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1021/2054] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:22 2022] POST /identity/v3/projects => generated 373 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:22.787674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:22.789111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:22.861856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25041a15-f324-4c6b-b394-aeb0daf2e9c7 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:22.924186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:22.924804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:22.925093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:22.925373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:22.926079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:22.928369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.210330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1034/2055] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:22 2022] POST /identity/v3/users => generated 287 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:23.211723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.214159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.216001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.226656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b894c-e2e0-4a9e-825b-656a42f94f60 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.232098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d35e-8c77-42cc-b457-f9aa25755759 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.233800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.234489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.234777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.235097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.235840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.237451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.237633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.239240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.239536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.239843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] PATH_INFO: `/v3/projects/5544755ae458477d88d411b080a2517a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.241106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5544755ae458477d88d411b080a2517a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.243502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.248855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1035/2056] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:23.253210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.263955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c285ec-0a18-4e89-ac09-bdc0bbe5958f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.269047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.269850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.270252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.270598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.271582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.274210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.292655 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1036/2057] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:23.306595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.307706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dde05-5fc5-4797-88b2-828ecce65c81 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.318384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.324943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.325287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.325539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] PATH_INFO: `/v3/projects/5f5c3fd01b334902bae81d9929cfa05a/users/9d69fb173f48483f9efef0cf6627e453/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.344611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5c3fd01b334902bae81d9929cfa05a, user_id=9d69fb173f48483f9efef0cf6627e453, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.344611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2d674-a184-4bac-b95f-1c78507761ac admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.530551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1037/2058] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:23 2022] PUT /identity/v3/projects/5f5c3fd01b334902bae81d9929cfa05a/users/9d69fb173f48483f9efef0cf6627e453/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:23.535845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.536065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.536684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.555638 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5544755ae458477d88d411b080a2517a', 'request_id': 'req-5db287a8-907d-442b-ae3f-5eada9f4be1b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:23.561838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5544755ae458477d88d411b080a2517a', 'request_id': 'req-5db287a8-907d-442b-ae3f-5eada9f4be1b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:23.561985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:23.563323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.639065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1038/2059] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:23.648255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.649567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.671983 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5544755ae458477d88d411b080a2517a', 'request_id': 'req-5db287a8-907d-442b-ae3f-5eada9f4be1b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:23.685785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] The token cache is being invalidate because project 5544755ae458477d88d411b080a2517a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:23.686153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:23.686970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5db287a8-907d-442b-ae3f-5eada9f4be1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:23.689445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1022/2060] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:23 2022] DELETE /identity/v3/projects/5544755ae458477d88d411b080a2517a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:23.695335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a9a0-1863-4f55-a8dc-06170b63b1af None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.695980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a9a0-1863-4f55-a8dc-06170b63b1af None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.697247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a9a0-1863-4f55-a8dc-06170b63b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.709473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1499d6-e312-4020-a990-20cdd8ee26eb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.721306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.722073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-39a1a9a0-1863-4f55-a8dc-06170b63b1af None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:23.722250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.722591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.722880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.724399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-39a1a9a0-1863-4f55-a8dc-06170b63b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.726629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.735043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1023/2061] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:23.739547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.741079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.750030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.750030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.750030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.754005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1039/2062] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:23.766829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:23.768128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:23.781521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1024/2063] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:23.783528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.795145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556abfd-8d25-4c45-b7da-437f60c91813 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.800144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.800809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.801099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.801379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.802107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:23.802418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.804054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.816036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1040/2064] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:23.820730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:23.830554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924242-65c1-43c1-864d-d2aded788ca2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:23.835377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:23.836045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:23.836336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:23.836618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:23.837370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:23.839332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:23.999486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1025/2065] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:23 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:24.000583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:24.014502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b348b5-b368-445f-982f-ad3fc9a21bd5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:24.073364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:24.074041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.074327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.074606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.075390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:24.077322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:24.111449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:24.269643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1041/2066] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:23 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:24.271328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:24.281884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e89e7-8547-45b4-b6ec-303789df1acc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:24.286566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:24.287501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.287657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.287825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.288569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:24.290175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:24.300837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1026/2067] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:24.304980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:24.316036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56da73b-444c-459d-8777-084a1bebfcc6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:24.320993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:24.321646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.321931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.322210 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.322944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:24.324552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59114-c426-41fd-af54-18013963d1a5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:24.335227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1042/2068] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:24.344169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:24.353978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545d993-c093-485f-b325-22babcdcd295 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:24.359071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:24.359941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.360259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.360542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] PATH_INFO: `/v3/projects/8222fbeae9cb440e945fc7ed10869c9d/users/0eb5105ff58e47518d390d5907a41804/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.375451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8222fbeae9cb440e945fc7ed10869c9d, user_id=0eb5105ff58e47518d390d5907a41804, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:24.378117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:24.461881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1027/2069] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:24 2022] PUT /identity/v3/projects/8222fbeae9cb440e945fc7ed10869c9d/users/0eb5105ff58e47518d390d5907a41804/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:24.472653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.472954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.473241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.505364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] MFA Rules not processed for user `0eb5105ff58e47518d390d5907a41804`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:24.506696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:24.585892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1043/2070] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:24.596661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:24.598405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:24.661252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b8834-94a9-4176-b07d-045107cbfe26 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:24.665098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:24.665745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:24.666034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:24.666317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:24.679683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:24.681169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47e084-4ed3-4086-ba94-55073ec64038 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:24.690740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1028/2071] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:24.907216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:25.149529 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:25.150937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:25.163612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b9428-76b0-4a80-a545-c7a040f907fc None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:25.167396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:25.173022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:25.173022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:25.173022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:25.173022 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:25.231794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:25.233246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:25.242406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1044/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:25.433405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd058-5553-4447-a419-299319aba718 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:25.434268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd058-5553-4447-a419-299319aba718 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:25.434268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd058-5553-4447-a419-299319aba718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:25.467761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1e8dd058-5553-4447-a419-299319aba718 None None] MFA Rules not processed for user `9d69fb173f48483f9efef0cf6627e453`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:25.469026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1e8dd058-5553-4447-a419-299319aba718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:25.539414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1029/2073] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:25.543881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:25.545934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:25.628445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adf1df-d8e5-43ff-9697-52507c943016 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:25.640318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:25.646339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:25.646339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:25.646339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:25.646339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:25.646339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:25.785427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1045/2074] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:25 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:25.786571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd058-5553-4447-a419-299319aba718 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:25.799095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd058-5553-4447-a419-299319aba718 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:25.862590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:25.863289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:25.863575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:25.863935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:25.864556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:25.866466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:25.955087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1030/2075] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:25 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:25.959018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:25.970104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062646-fbaa-4363-9560-bf507f93f1e5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:25.974834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:25.975546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:25.975832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:25.976179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:25.977114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:25.978875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:25.990069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1046/2076] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:25.994151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:26.004842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6d683-1b60-4892-a8e0-a7b00ef80d02 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:26.009433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:26.010332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:26.010617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:26.010893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] PATH_INFO: `/v3/projects/937fca4a7a264317a42fd6c1700b15b3/users/a36a8a56df3c43c290e7020109cbeb99/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:26.025365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=937fca4a7a264317a42fd6c1700b15b3, user_id=a36a8a56df3c43c290e7020109cbeb99, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:26.028042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:26.140051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1031/2077] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:25 2022] PUT /identity/v3/projects/937fca4a7a264317a42fd6c1700b15b3/users/a36a8a56df3c43c290e7020109cbeb99/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:26.144214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:26.154404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3d4ca-a765-4d4f-8737-32fd2c487039 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:26.166793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:26.224443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:26.225124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:26.225412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:26.225691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:26.226420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:26.228055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:26.239244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1047/2078] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:26.248760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:26.259165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4fbd8-5b21-4c8a-bffd-9c481a2d96bc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:26.263765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:26.264628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:26.264910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:26.265184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] PATH_INFO: `/v3/projects/937fca4a7a264317a42fd6c1700b15b3/users/a36a8a56df3c43c290e7020109cbeb99/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:26.267197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937fca4a7a264317a42fd6c1700b15b3, user_id=a36a8a56df3c43c290e7020109cbeb99, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:26.269970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43fe38-5f76-4cd4-8426-a54f93541ea5 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:26.280694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1032/2079] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:26 2022] PUT /identity/v3/projects/937fca4a7a264317a42fd6c1700b15b3/users/a36a8a56df3c43c290e7020109cbeb99/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:27.820523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:27.878875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bffa5-50d6-4b05-88ee-a28545776091 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:27.931914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:27.932516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:27.932890 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:27.933169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:27.935050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:28.010913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:28.016919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:28.023849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1048/2080] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:28.115156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:28.127316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:28.127613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:28.128279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:28.159874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] MFA Rules not processed for user `a36a8a56df3c43c290e7020109cbeb99`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:28.162481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:28.237950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1033/2081] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:28.273365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:28.283315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e767-32f9-4868-95ba-a7c5ae433f82 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:28.287749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:28.288521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:28.288805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:28.289088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:28.348455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:28.350330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fa646-6fb2-43fb-9455-7c798e21108e service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:28.361550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1049/2082] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:28.503198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:29.526181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:30.544652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:30.546654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:30.625948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76191507-661e-4c1a-9f57-d85c35050df6 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:30.634412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:30.635171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:30.635452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:30.635799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] PATH_INFO: `/v3/users/ec0d24d057ce40d0b7618a8673df7b95` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:30.637279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0d24d057ce40d0b7618a8673df7b95)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:30.640330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:30.925944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:30.990559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0d24d057ce40d0b7618a8673df7b95'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:30.991247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0d24d057ce40d0b7618a8673df7b95'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:30.997962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0d24d057ce40d0b7618a8673df7b95'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:31.168797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0d24d057ce40d0b7618a8673df7b95'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:31.184525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0d24d057ce40d0b7618a8673df7b95'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:31.189244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1034/2083] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:30 2022] DELETE /identity/v3/users/ec0d24d057ce40d0b7618a8673df7b95 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:31.193831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:31.194312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:31.194418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:31.212838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:31.214114 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:31.274703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1050/2084] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:31.284415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:31.285843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:31.348468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af5b61-2394-4476-870d-f4249db053cf admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:31.353471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:31.354745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:31.355179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:31.355606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:31.370464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:31.371967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:31.382968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1035/2085] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:31.429482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:31.431369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:31.503515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457a2e1-8c02-4519-bd64-7d6fb17cc10a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:31.508590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:31.509246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:31.509528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:31.509807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:31.511615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:31.581217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:31.582663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291b4e2-0aa8-4562-8b71-654ea318cbba service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:31.592570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1051/2086] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:32.001350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:32.002744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:32.020120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128edfdd-453b-4335-899d-e7c0a02b22b0 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:32.025341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:32.026197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.026477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.026754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] PATH_INFO: `/v3/projects/e64a8113a009471d8be65fbc35588f69` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.027961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64a8113a009471d8be65fbc35588f69)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:32.030414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:32.234794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64a8113a009471d8be65fbc35588f69', 'request_id': 'req-0b6ff2eb-f783-4132-acc5-46b005435e0e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:32.240663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64a8113a009471d8be65fbc35588f69', 'request_id': 'req-0b6ff2eb-f783-4132-acc5-46b005435e0e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:32.475408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64a8113a009471d8be65fbc35588f69', 'request_id': 'req-0b6ff2eb-f783-4132-acc5-46b005435e0e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:32.490184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] The token cache is being invalidate because project e64a8113a009471d8be65fbc35588f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:32.490184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:32.490341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b6ff2eb-f783-4132-acc5-46b005435e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:32.492006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1036/2087] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:31 2022] DELETE /identity/v3/projects/e64a8113a009471d8be65fbc35588f69 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:32.497109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.497595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.497677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.520866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:32.522403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:32.588949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:32.606374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1052/2088] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:32.615164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9beca-79dc-4609-b473-bf3706710399 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.615461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9beca-79dc-4609-b473-bf3706710399 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.615739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9beca-79dc-4609-b473-bf3706710399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.635257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-add9beca-79dc-4609-b473-bf3706710399 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:32.636481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-add9beca-79dc-4609-b473-bf3706710399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:32.645140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1037/2089] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:32.649365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:32.659010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571d2a7-e4b2-4933-8b49-5e5f1e3a45f5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:32.663899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:32.664720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.665033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.665312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.666085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:32.666401 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:32.668108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:32.684378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1053/2090] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:32.686119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9beca-79dc-4609-b473-bf3706710399 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:32.695605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9beca-79dc-4609-b473-bf3706710399 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:32.700766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:32.702031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.702350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.702630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.703365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:32.705458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:32.889840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1038/2091] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:32 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:32.893207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:32.915471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ca4ed-8fd9-4e53-bbce-7efbb756e2fc admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:32.977246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:32.978004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:32.978430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:32.978794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:32.979601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:32.981957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:33.115775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1054/2092] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:32 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:33.119826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:33.130602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f5f95-9374-4e53-b0fc-7b882f6eb62d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:33.135184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:33.135839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.136132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.136412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.137141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:33.138770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:33.150050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1039/2093] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:33.154113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:33.165460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f99a-de85-4c0b-bf73-09b946bd85d1 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:33.172473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:33.173316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.173609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.173893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.174632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:33.176316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f5038-c59e-48f9-b7d4-23ea35f1c242 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:33.193091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1055/2094] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:33.194027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:33.204421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e75162-42f1-4dc2-b5df-7c6bd09301d7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:33.209156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:33.210049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.210328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.210602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] PATH_INFO: `/v3/projects/b3ed8d86b3b340e0a6b38a96147ca6b3/users/829a5814b247429f8bcb169320f2a1e8/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.227661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ed8d86b3b340e0a6b38a96147ca6b3, user_id=829a5814b247429f8bcb169320f2a1e8, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:33.230290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:33.432405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1040/2095] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:33 2022] PUT /identity/v3/projects/b3ed8d86b3b340e0a6b38a96147ca6b3/users/829a5814b247429f8bcb169320f2a1e8/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:33.442650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.443063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.443429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.471837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] MFA Rules not processed for user `829a5814b247429f8bcb169320f2a1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:33.473100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:33.548033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1056/2096] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:33.599225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:33.600956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:33.666670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393616b-a0df-4727-9b97-f720f28fb507 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:33.671159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:33.672046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.672499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.672943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.683268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:33.686742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:33.704304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1041/2097] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:33.732127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:33.795686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7e1f1-a5b1-453a-b0e4-cec109acfb0f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:33.800571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:33.801588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:33.801878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:33.802160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] PATH_INFO: `/v3/users/829a5814b247429f8bcb169320f2a1e8` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:33.803474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=829a5814b247429f8bcb169320f2a1e8)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:33.805402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:34.029271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:34.031283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.045508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef73ba-269c-49cb-a588-a5eb4ca35ece service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:34.050482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:34.059546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:34.059546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:34.059546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] PATH_INFO: `/v3/users/d218157c531a4e4c9900649d175ed249` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:34.059546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d218157c531a4e4c9900649d175ed249)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:34.059546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:34.159895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829a5814b247429f8bcb169320f2a1e8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.160464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829a5814b247429f8bcb169320f2a1e8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.168878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829a5814b247429f8bcb169320f2a1e8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.297042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829a5814b247429f8bcb169320f2a1e8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.309709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-37e2a607-c70a-4df8-9249-5df66af660f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829a5814b247429f8bcb169320f2a1e8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.318047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1057/2098] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:33 2022] DELETE /identity/v3/users/829a5814b247429f8bcb169320f2a1e8 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:34.324505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:34.324800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:34.325078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:34.343397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:34.344756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.404391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1058/2099] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:34.414451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:34.415324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd218157c531a4e4c9900649d175ed249'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.415965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.416546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd218157c531a4e4c9900649d175ed249'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.425222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd218157c531a4e4c9900649d175ed249'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.480320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d3b06-cea2-4843-921e-dcfa0d3b9ce2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:34.485698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:34.486905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:34.486905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:34.486905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:34.489293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.502142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:34.503597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950117-4195-41ad-a978-791b4e0664f8 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:34.513154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1059/2100] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:34.557459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd218157c531a4e4c9900649d175ed249'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.572162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd218157c531a4e4c9900649d175ed249'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:34.578292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1042/2101] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:34 2022] DELETE /identity/v3/users/d218157c531a4e4c9900649d175ed249 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:34.582795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:34.583292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:34.583407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:34.601243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:34.602447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.662595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1060/2102] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:34.673912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:34.675418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:34.733993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cadd3-64e7-4264-b28e-a88e22c2bd4e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:34.737804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:34.743644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:34.743644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:34.743644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:34.753710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:34.755218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:34.766344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1043/2103] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:34.798052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:35.096004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:35.304293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:35.306329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:35.323378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1701b-187b-4e1e-8426-19f59b750d4e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:35.328835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:35.330106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.331773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.332478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] PATH_INFO: `/v3/projects/b3ed8d86b3b340e0a6b38a96147ca6b3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.334500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ed8d86b3b340e0a6b38a96147ca6b3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:35.343877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:35.530357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ed8d86b3b340e0a6b38a96147ca6b3', 'request_id': 'req-39c0358c-9df2-4b18-817d-8430515f3fab'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.535841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ed8d86b3b340e0a6b38a96147ca6b3', 'request_id': 'req-39c0358c-9df2-4b18-817d-8430515f3fab'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.670043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ed8d86b3b340e0a6b38a96147ca6b3', 'request_id': 'req-39c0358c-9df2-4b18-817d-8430515f3fab'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.676532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:35.687456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:35.687555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] The token cache is being invalidate because project b3ed8d86b3b340e0a6b38a96147ca6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:35.687555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.687555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-39c0358c-9df2-4b18-817d-8430515f3fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.693485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1061/2104] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:35 2022] DELETE /identity/v3/projects/b3ed8d86b3b340e0a6b38a96147ca6b3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:35.693660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953916d-21a0-4b41-bb0b-78cdf24a60d1 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:35.695666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a978b49-cb12-46a5-b166-dccd0a510bd5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.696054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a978b49-cb12-46a5-b166-dccd0a510bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.696339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a978b49-cb12-46a5-b166-dccd0a510bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.718000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-9a978b49-cb12-46a5-b166-dccd0a510bd5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:35.719434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9a978b49-cb12-46a5-b166-dccd0a510bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:35.752979 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:35.753842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.754123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.754399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] PATH_INFO: `/v3/projects/7685b2d9079d4073b4c8013cfe9a7cd3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.755628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7685b2d9079d4073b4c8013cfe9a7cd3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:35.757537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:35.781163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1062/2105] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:35.788772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.789100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.789395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.807459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:35.808690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:35.817186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1063/2106] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:35.821330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:35.830461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec477615-0ba9-4767-b381-a65b51d324e5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:35.835207 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:35.835836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.836152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.836435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.837178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:35.837491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:35.839153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:35.850504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1064/2107] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:35.854511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:35.863917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae460a3-4dd6-4c33-a04f-410b3fe9c561 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:35.868557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:35.869172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:35.869458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:35.869740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:35.870411 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:35.872284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:35.977706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7685b2d9079d4073b4c8013cfe9a7cd3', 'request_id': 'req-9c49eebc-6f77-48ca-8498-88e45477abc4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:35.983952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7685b2d9079d4073b4c8013cfe9a7cd3', 'request_id': 'req-9c49eebc-6f77-48ca-8498-88e45477abc4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.002195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1065/2108] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:35 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:36.003127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.083453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cd54b-76f6-49b2-a5df-8cb0071aaa71 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.090811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.091527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.091848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.092142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.092825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.094739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.128771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7685b2d9079d4073b4c8013cfe9a7cd3', 'request_id': 'req-9c49eebc-6f77-48ca-8498-88e45477abc4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.149517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] The token cache is being invalidate because project 7685b2d9079d4073b4c8013cfe9a7cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:36.149517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.149517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.149517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1044/2109] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:35 2022] DELETE /identity/v3/projects/7685b2d9079d4073b4c8013cfe9a7cd3 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:36.150093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.151442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:36.221392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49eebc-6f77-48ca-8498-88e45477abc4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.225993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.226868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.227185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.227463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] PATH_INFO: `/v3/users/e9ebc6c9b6e54f5a89b3709d374c6880` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.236970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ebc6c9b6e54f5a89b3709d374c6880)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.236970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.268007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1066/2110] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:36 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:36.272069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.273695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:36.285657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fb1a-8f8d-4129-ad88-0fa0210768d4 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.290262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.290998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.291286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.291564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.292325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.293902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.305188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1067/2111] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:36.310501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.320769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d85da1-e552-4173-8a17-7fdef5933b85 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.325440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.326129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.326416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.326696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.327457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.329033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.339674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1068/2112] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:36.344697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.357468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e80bd-58c9-4ead-9c04-93a0784f68fe admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.363021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.364044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.364433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.370504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] PATH_INFO: `/v3/projects/da06c03197114259aac4bf5444485d51/users/a7fd310fe0254401b10f6b1517994233/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.380895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da06c03197114259aac4bf5444485d51, user_id=a7fd310fe0254401b10f6b1517994233, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.383710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3386957-e5fc-4877-bdcc-34bf82075b29 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.493345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1069/2113] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:36 2022] PUT /identity/v3/projects/da06c03197114259aac4bf5444485d51/users/a7fd310fe0254401b10f6b1517994233/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:36.503042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.503042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.503042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.503313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ebc6c9b6e54f5a89b3709d374c6880'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.503313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ebc6c9b6e54f5a89b3709d374c6880'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.510119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ebc6c9b6e54f5a89b3709d374c6880'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.514896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:36.516137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:36.574097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1070/2114] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:36.584585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:36.585935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:36.640636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ebc6c9b6e54f5a89b3709d374c6880'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.643418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bb6c-9f35-4636-9c47-71394ae2b5de None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:36.648384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:36.649049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:36.649337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:36.649616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:36.654502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ebc6c9b6e54f5a89b3709d374c6880'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:36.659394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:36.660632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1045/2115] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:36 2022] DELETE /identity/v3/users/e9ebc6c9b6e54f5a89b3709d374c6880 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:36.660765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:36.670506 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1071/2116] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:37.424320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:37.425774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:37.513805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f51d8-7ddb-497e-bcf4-48f918c3649a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:37.518455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:37.519344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:37.519629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:37.519907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] PATH_INFO: `/v3/projects/af3d25e97bdf43d7bcf595728698e115` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:37.521090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3d25e97bdf43d7bcf595728698e115)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:37.523050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:37.527820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:37.535124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:37.545325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f264d3-fba3-48d7-9d44-e5b7c4eb30b9 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:37.553814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:37.554704 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:37.555026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:37.555312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] PATH_INFO: `/v3/users/0eb5105ff58e47518d390d5907a41804` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:37.556523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb5105ff58e47518d390d5907a41804)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:37.558469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:37.706916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:37.725947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3d25e97bdf43d7bcf595728698e115', 'request_id': 'req-c7b00b28-b907-44ef-85c0-923ffe1a0778'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.732748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3d25e97bdf43d7bcf595728698e115', 'request_id': 'req-c7b00b28-b907-44ef-85c0-923ffe1a0778'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.865478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3d25e97bdf43d7bcf595728698e115', 'request_id': 'req-c7b00b28-b907-44ef-85c0-923ffe1a0778'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.879930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] The token cache is being invalidate because project af3d25e97bdf43d7bcf595728698e115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:37.880222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.880436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb5105ff58e47518d390d5907a41804'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.881063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.881728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb5105ff58e47518d390d5907a41804'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.885210 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1046/2117] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:37 2022] DELETE /identity/v3/projects/af3d25e97bdf43d7bcf595728698e115 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:37.887840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:37.890289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb5105ff58e47518d390d5907a41804'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:37.891223 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:37.957277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00b28-b907-44ef-85c0-923ffe1a0778 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:37.962229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:37.963152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:37.963438 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:37.963717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] PATH_INFO: `/v3/users/1a1d6a8d1ac24b1a8ed2887b37a90d57` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:37.964933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1d6a8d1ac24b1a8ed2887b37a90d57)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:37.967410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:38.046331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb5105ff58e47518d390d5907a41804'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.058843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bc4556b1-bd3e-4bac-aa91-8f129313e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb5105ff58e47518d390d5907a41804'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.064853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1072/2118] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:37 2022] DELETE /identity/v3/users/0eb5105ff58e47518d390d5907a41804 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:38.069036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:38.069327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:38.069860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:38.086645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:38.087881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:38.147094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1073/2119] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:38.161605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:38.164123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:38.237543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4acc-ea12-4166-be60-a716e1a594dd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:38.242717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:38.243859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:38.243859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:38.244084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:38.244990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1d6a8d1ac24b1a8ed2887b37a90d57'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.245742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1d6a8d1ac24b1a8ed2887b37a90d57'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.253981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:38.254524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1d6a8d1ac24b1a8ed2887b37a90d57'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.256350 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f1306-f976-4c2b-b4ca-9a5f9150b23b service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:38.268698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1074/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:38.448879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1d6a8d1ac24b1a8ed2887b37a90d57'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.463245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1d6a8d1ac24b1a8ed2887b37a90d57'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:38.473380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1047/2121] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:37 2022] DELETE /identity/v3/users/1a1d6a8d1ac24b1a8ed2887b37a90d57 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:38.891786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:38.905091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:38.905643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:38.905643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:38.935881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] MFA Rules not processed for user `a7fd310fe0254401b10f6b1517994233`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:38.937213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:38.964473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:38.966474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.010689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1075/2122] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:39.021562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.022958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.042145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86f5b4-35f2-400a-9769-62d032d95877 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.047503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.048339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.048628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.048907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] PATH_INFO: `/v3/projects/8222fbeae9cb440e945fc7ed10869c9d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.050156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8222fbeae9cb440e945fc7ed10869c9d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.054594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.081180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09bacf-6bfb-47f5-a8cd-a859b9fde94c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.084851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.085559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.085777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.086051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.102084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.102084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.106998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1076/2123] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:39.118717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.120681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.142400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bf493-a86b-48e4-9a45-6c09a6544207 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.147885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.149262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.149684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.150101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] PATH_INFO: `/v3/projects/f47539aa5ebb472c89aaf74eb5106d07` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.151373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47539aa5ebb472c89aaf74eb5106d07)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.153316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.235898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8222fbeae9cb440e945fc7ed10869c9d', 'request_id': 'req-54da7469-b037-440d-8d6f-bccf14852899'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.240426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8222fbeae9cb440e945fc7ed10869c9d', 'request_id': 'req-54da7469-b037-440d-8d6f-bccf14852899'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.332106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47539aa5ebb472c89aaf74eb5106d07', 'request_id': 'req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.337381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47539aa5ebb472c89aaf74eb5106d07', 'request_id': 'req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.357400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8222fbeae9cb440e945fc7ed10869c9d', 'request_id': 'req-54da7469-b037-440d-8d6f-bccf14852899'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.369394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] The token cache is being invalidate because project 8222fbeae9cb440e945fc7ed10869c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:39.369682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.370332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-54da7469-b037-440d-8d6f-bccf14852899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.371966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1048/2124] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:38 2022] DELETE /identity/v3/projects/8222fbeae9cb440e945fc7ed10869c9d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:39.378090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7601e-f2ee-40c7-b2e8-64c6c38b07d3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.378375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7601e-f2ee-40c7-b2e8-64c6c38b07d3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.378652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7601e-f2ee-40c7-b2e8-64c6c38b07d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.396624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-9fe7601e-f2ee-40c7-b2e8-64c6c38b07d3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:39.397867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9fe7601e-f2ee-40c7-b2e8-64c6c38b07d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.457013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1049/2125] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:39.464520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.464809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.465088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.485620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:39.486982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.495196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1050/2126] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:39.499218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.508141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b1fcc-750e-4920-917d-9f19eb1c92b8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.508810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47539aa5ebb472c89aaf74eb5106d07', 'request_id': 'req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.514435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:39.514776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.514776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.514776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.514776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.515147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:39.515245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.516843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.526261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] The token cache is being invalidate because project f47539aa5ebb472c89aaf74eb5106d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:39.526889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.527650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a49f50a9-43a2-4dc5-bc4d-f7b07fa003bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:39.529834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1077/2127] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:39 2022] DELETE /identity/v3/projects/f47539aa5ebb472c89aaf74eb5106d07 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:39.530190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1051/2128] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:39.534248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.534769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33326a0-6c8b-43dc-b84d-8874b439db14 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.535096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33326a0-6c8b-43dc-b84d-8874b439db14 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.535381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33326a0-6c8b-43dc-b84d-8874b439db14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.536125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.549691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31876-5e62-4f0c-a5af-831146b20d8c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.558648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.558648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.558648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.558648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.559027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.559027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.568193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d33326a0-6c8b-43dc-b84d-8874b439db14 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:39.569660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d33326a0-6c8b-43dc-b84d-8874b439db14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.580758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1078/2129] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:39.589417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.589799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.590150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.609891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:39.611129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:39.620063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1079/2130] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:39.624408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.639639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aad7f-5fb0-4726-80b4-7092e2b3f3cd None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.639639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.639639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.639639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.640031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.640031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:39.640161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.641719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.656076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1080/2131] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:39.660417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.672034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd126a2-95de-4b9f-b924-2b0aa927a823 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.677191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.677857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.678146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.678422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.679156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.681305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.738417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1052/2132] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:39.742582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.803355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ed61d-20a4-4263-a172-44aad42c24fc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:39.808462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:39.809133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:39.809421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:39.809694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:39.810521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:39.812631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:39.935788 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1081/2133] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/projects => generated 371 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:39.937192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.965650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1053/2134] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:39.971931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:39.987320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcb3eb-1887-4342-b773-1ff8dcdd2ede admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.004985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58544cbb-1179-4118-bada-18e6fa5f82e3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.010127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.011231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.011526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.011811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.012517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.015287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.057833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.058580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.067641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.067927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.068899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.070829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.081787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1054/2135] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:40.085722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.096536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d8cd-bbd0-4ad8-a4bf-5fae181f253c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.101364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.102023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.102312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.102595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.103359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.105033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.116144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1055/2136] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:40.120238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.130488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a64f8-6569-4764-9aa4-8b2dda79198e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.135279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.136158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.136441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.136716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] PATH_INFO: `/v3/projects/724dea023b2d456ca81b4deccfee4c39/users/93163c5e89274773a3f1e460c8d17909/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.139560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1082/2137] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:39 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:40.151678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724dea023b2d456ca81b4deccfee4c39, user_id=93163c5e89274773a3f1e460c8d17909, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.154380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673d28-e512-4cfe-91e8-974bb8672278 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.154616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.165246 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0819ade-1ffe-4cc9-887e-8ac374c0de25 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.171195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.171891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.172173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.172454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.173188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.174953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.190761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1083/2138] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:40.194840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.204588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1f846-585b-4df5-8889-228967e242b4 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.209877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.210043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.210338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.210620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.211379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.212964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.224396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1084/2139] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:45:40.228165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.238420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa6565-1972-4db9-a717-ac9f6355f158 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.243351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.244510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.244800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.245079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] PATH_INFO: `/v3/projects/7ca93a40aa85448a8e5165f4c268678a/users/61d74fdb719b40208ee408b838256a0f/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.262775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca93a40aa85448a8e5165f4c268678a, user_id=61d74fdb719b40208ee408b838256a0f, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.265841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85b9a9-b49d-41c6-b13e-a21b76b8a25c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.298595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1056/2140] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:40 2022] PUT /identity/v3/projects/724dea023b2d456ca81b4deccfee4c39/users/93163c5e89274773a3f1e460c8d17909/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:40.309067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.309473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.310753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.339320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] MFA Rules not processed for user `93163c5e89274773a3f1e460c8d17909`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:40.340600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:40.403577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1057/2141] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:40.408046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.470005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8930e-cfa1-44ec-b5d3-dd152228a3b1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.474893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.485108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.485108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.485108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.485108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.485674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1085/2142] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:40 2022] PUT /identity/v3/projects/7ca93a40aa85448a8e5165f4c268678a/users/61d74fdb719b40208ee408b838256a0f/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:40.485778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.485865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.485865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.485865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.506383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:40.509740 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:40.569827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1086/2143] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:40.585724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.587086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:40.645550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a13bf-8980-4fc0-9838-6563b18e9b53 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.649553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.650212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.650490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.650772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.663421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1058/2144] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:40 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:40.664501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.666464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.667458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.679150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1087/2145] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:40.737789 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7d117-7110-44c9-9a82-d8651e2b969f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.743200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.746916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.746916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.746916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.746916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.747392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.900882 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1059/2146] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:40 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:40.905503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.915813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6d1f2-4b8c-4726-8342-ad0b11a21133 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.921386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.922199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.922362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.922641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.923396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.925002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:40.939124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1088/2147] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:40.940348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:40.951185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182d9d-8d0c-4da0-b869-357a7ad2a9cf admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:40.955902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:40.956772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:40.957060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:40.957366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] PATH_INFO: `/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962/users/6d6a4f51ff104ba385aa748add199c2e/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:40.973505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb72cab512eb40cbaa3d5e25c65ae962, user_id=6d6a4f51ff104ba385aa748add199c2e, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:40.976593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baeeaf-8396-4316-8d24-966ebb24985a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:41.098846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1060/2148] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:40 2022] PUT /identity/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962/users/6d6a4f51ff104ba385aa748add199c2e/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:41.111442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:41.124933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b4268-c462-4fd4-9547-5b811df81760 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:41.168556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a2d4-744c-40f9-b75d-c061cb388bd8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:41.168857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a2d4-744c-40f9-b75d-c061cb388bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:41.169146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a2d4-744c-40f9-b75d-c061cb388bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:41.201920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d589a2d4-744c-40f9-b75d-c061cb388bd8 None None] MFA Rules not processed for user `a7fd310fe0254401b10f6b1517994233`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:41.203221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d589a2d4-744c-40f9-b75d-c061cb388bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:41.235643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:41.236450 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:41.236800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:41.237144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:41.237878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:41.239947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:41.250649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1089/2149] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:41.254420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:41.267334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f5f4-9e3d-4da3-b6e2-eb9260907bb2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:41.272482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:41.273519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:41.273802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:41.274074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] PATH_INFO: `/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962/users/6d6a4f51ff104ba385aa748add199c2e/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:41.276163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1061/2150] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:41.276392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb72cab512eb40cbaa3d5e25c65ae962, user_id=6d6a4f51ff104ba385aa748add199c2e, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:41.279222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:41.295958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1090/2151] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:41 2022] PUT /identity/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962/users/6d6a4f51ff104ba385aa748add199c2e/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:41.309910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf3b7-ae09-4e24-900d-cbdf495bc25b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:41.310034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf3b7-ae09-4e24-900d-cbdf495bc25b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:41.310873 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf3b7-ae09-4e24-900d-cbdf495bc25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:41.338701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-82fbf3b7-ae09-4e24-900d-cbdf495bc25b None None] MFA Rules not processed for user `6d6a4f51ff104ba385aa748add199c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:41.346609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-82fbf3b7-ae09-4e24-900d-cbdf495bc25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:41.407806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:41.412457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1062/2152] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:41.425762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:41.429093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:41.489887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75af1-1798-4fa2-937d-9e63e4ace9c6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:41.493398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:41.494244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:41.494523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:41.494909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:41.505893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:41.507729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:41.530810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1091/2153] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:42.545590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:42.697821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:42.698142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:42.698426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:42.727103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] MFA Rules not processed for user `61d74fdb719b40208ee408b838256a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:42.728343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:42.794197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1063/2154] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:43.016725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:43.025282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a098a-a551-45be-9862-b19650206416 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:43.029645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:43.030292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:43.030576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:43.030855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:43.039745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:43.041584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3490561-ba17-4972-82a1-622716e1b5a3 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:43.050755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1092/2155] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:44.551516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:44.755568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:44.816833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b29cd-ab57-45d4-a9c5-e174cdbe5479 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:44.821647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:44.822493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:44.822779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:44.823093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] PATH_INFO: `/v3/users/93163c5e89274773a3f1e460c8d17909` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:44.824279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93163c5e89274773a3f1e460c8d17909)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:44.826757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:45.096519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93163c5e89274773a3f1e460c8d17909'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:45.097236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93163c5e89274773a3f1e460c8d17909'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:45.103996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93163c5e89274773a3f1e460c8d17909'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:45.212196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face4106-6091-4b58-b3b6-d36a09f449d3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:45.212487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face4106-6091-4b58-b3b6-d36a09f449d3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:45.212820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face4106-6091-4b58-b3b6-d36a09f449d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:45.230888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-face4106-6091-4b58-b3b6-d36a09f449d3 None None] MFA Rules not processed for user `61d74fdb719b40208ee408b838256a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:45.232670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-face4106-6091-4b58-b3b6-d36a09f449d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:45.240167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1093/2156] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:45:45.249489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93163c5e89274773a3f1e460c8d17909'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:45.259825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93163c5e89274773a3f1e460c8d17909'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:45.266581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1064/2157] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:44 2022] DELETE /identity/v3/users/93163c5e89274773a3f1e460c8d17909 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:45.271208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:45.271485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:45.271771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:45.289824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:45.291061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:45.352364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1094/2158] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:45.362955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:45.364372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:45.451890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e52d47-0a4b-4b30-8c5a-8a40855eba31 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:45.455709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:45.456366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:45.456717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:45.457050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:45.468101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:45.469755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:45.482126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1065/2159] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:46.187225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:46.187225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:46.200035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542416b-42d8-4ff2-ae2a-af12e254858c None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:46.216158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:46.216158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:46.216158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:46.216158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] PATH_INFO: `/v3/projects/724dea023b2d456ca81b4deccfee4c39` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:46.216551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=724dea023b2d456ca81b4deccfee4c39)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:46.216551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:46.385227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724dea023b2d456ca81b4deccfee4c39', 'request_id': 'req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.392567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724dea023b2d456ca81b4deccfee4c39', 'request_id': 'req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.512174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724dea023b2d456ca81b4deccfee4c39', 'request_id': 'req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.531588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] The token cache is being invalidate because project 724dea023b2d456ca81b4deccfee4c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:46.532287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.533168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.538983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1095/2160] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:46 2022] DELETE /identity/v3/projects/724dea023b2d456ca81b4deccfee4c39 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:46.539980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:46.551231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18131f-7dd2-423a-b116-bd807e4cee4d service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:46.649491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:46.652008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:46.652304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:46.652591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] PATH_INFO: `/v3/users/6d6a4f51ff104ba385aa748add199c2e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:46.653949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6a4f51ff104ba385aa748add199c2e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:46.656626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:46.745859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:46.942348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6a4f51ff104ba385aa748add199c2e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.943049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6a4f51ff104ba385aa748add199c2e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:46.949986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6a4f51ff104ba385aa748add199c2e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:47.051978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6a4f51ff104ba385aa748add199c2e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:47.083509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6a4f51ff104ba385aa748add199c2e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:47.090943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1066/2161] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:46 2022] DELETE /identity/v3/users/6d6a4f51ff104ba385aa748add199c2e => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:47.645254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:47.647522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:47.754389 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729ed08-b5d1-4a9a-8fc1-751cdb74b69e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:47.814481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:47.815545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:47.815832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:47.816141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] PATH_INFO: `/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:47.817393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb72cab512eb40cbaa3d5e25c65ae962)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:47.819737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:47.976103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb72cab512eb40cbaa3d5e25c65ae962', 'request_id': 'req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:47.985793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb72cab512eb40cbaa3d5e25c65ae962', 'request_id': 'req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.105648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:48.107523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb72cab512eb40cbaa3d5e25c65ae962', 'request_id': 'req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.116145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:48.120538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] The token cache is being invalidate because project fb72cab512eb40cbaa3d5e25c65ae962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:48.121059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.122581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e7b66f3b-72e1-49cf-a923-39b3ecc5b159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.130278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1096/2162] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:47 2022] DELETE /identity/v3/projects/fb72cab512eb40cbaa3d5e25c65ae962 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:48.132083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c977-bb87-4f55-aefe-d2f3da32decc None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.132721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c977-bb87-4f55-aefe-d2f3da32decc None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.133192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c977-bb87-4f55-aefe-d2f3da32decc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.193827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1ca8c977-bb87-4f55-aefe-d2f3da32decc None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:48.196135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1ca8c977-bb87-4f55-aefe-d2f3da32decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:48.211042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d39cf-fb32-499e-9b16-8b04c9a4ba52 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:48.262644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1097/2163] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:48.268081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:48.270405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.270749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.271077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.286332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:48.287769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.288380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.288848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] PATH_INFO: `/v3/users/8913d20bc0794f29afc75dd7300903e0` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.290521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8913d20bc0794f29afc75dd7300903e0)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:48.292838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:48.296668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:48.298666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:48.307975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1098/2164] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:48.313505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:48.328364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63dd92-359d-43e8-9bca-5cb620650aa3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:48.333578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:48.334295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.334581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.334859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.335731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:48.336061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:48.337881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:48.448047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1099/2165] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:45:48.448047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:48.448047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b1c4a-8ffc-48d1-986a-a70d608e2c90 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:48.448047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:48.449508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.449508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.449508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.449508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:48.449508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:48.844142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1100/2166] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:48 2022] POST /identity/v3/projects => generated 377 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:48.848282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:48.905322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8913d20bc0794f29afc75dd7300903e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.906043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8913d20bc0794f29afc75dd7300903e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.918913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8913d20bc0794f29afc75dd7300903e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:48.924400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf46d-7b3c-451e-9a89-6d5192dc6660 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:48.943229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:48.943229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:48.943229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:48.943229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:48.944055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:48.948176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.048425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8913d20bc0794f29afc75dd7300903e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:49.057032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1101/2167] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:48 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:49.061220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:49.067186 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-58f37938-f5bf-42cc-a827-f0b9c4ae6264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8913d20bc0794f29afc75dd7300903e0'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:49.067346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:49.074390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1067/2168] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:48 2022] DELETE /identity/v3/users/8913d20bc0794f29afc75dd7300903e0 => generated 0 bytes in 969 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:49.079732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.080217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.080501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.088660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de86d-02d2-455c-9d2a-9cad61fbd93a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:49.098381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:49.099617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:49.144392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:49.145068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.145359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.145642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.146381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:49.148023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.157285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1068/2169] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:49.158982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1102/2170] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:49.163965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:49.167391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:49.169064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:49.174765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a1245-1699-459a-acb3-28ef07a22e8b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:49.179736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:49.185201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.185201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.185201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.185201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:49.185201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.197766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1069/2171] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:49.198675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:49.212272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d1404-1fca-4b1f-bd15-01a8b11af000 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:49.217916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:49.218587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.218911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.219239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] PATH_INFO: `/v3/projects/bf2ab66fdf4c44bb9eec4d1875726b6d/users/5034fbc8f3dd464ea2d6ebb48457ff8e/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.237090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2ab66fdf4c44bb9eec4d1875726b6d, user_id=5034fbc8f3dd464ea2d6ebb48457ff8e, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:49.240127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.290532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434b8b-9ea1-403e-a508-9c8846538a62 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:49.295100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:49.295881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.296240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.296591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.316227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:49.316227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b62e60-2470-4c71-a174-84fe4d23860a service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.317849 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1103/2172] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:45:49.402202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1070/2173] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:49 2022] PUT /identity/v3/projects/bf2ab66fdf4c44bb9eec4d1875726b6d/users/5034fbc8f3dd464ea2d6ebb48457ff8e/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:49.404518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.404864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.405169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.433761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] MFA Rules not processed for user `5034fbc8f3dd464ea2d6ebb48457ff8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:49.435142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:49.507130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1104/2174] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:49.519590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:49.521257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:49.578248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7ab1-6872-4c31-9e28-f2616a1539c4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:49.581833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:49.582484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:49.583420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:49.583719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:49.621879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:49.623401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c214647-d9ed-4779-9a6f-e36f8ed30e10 service glance] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:49.633935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1071/2175] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:50.079515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:50.081176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:50.187138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4f4d2-1409-46b4-80c9-f1bff7b97216 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:50.192436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:50.193355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:50.193639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:50.193912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] PATH_INFO: `/v3/projects/4f2c3307a1e94d99a5f6426aefbdd8f8` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:50.195434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f2c3307a1e94d99a5f6426aefbdd8f8)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:50.206910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:50.344760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f2c3307a1e94d99a5f6426aefbdd8f8', 'request_id': 'req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:50.349963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f2c3307a1e94d99a5f6426aefbdd8f8', 'request_id': 'req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:50.476165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f2c3307a1e94d99a5f6426aefbdd8f8', 'request_id': 'req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:50.488425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] The token cache is being invalidate because project 4f2c3307a1e94d99a5f6426aefbdd8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:50.488725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:50.489359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-8cf2f2b4-77d5-4229-a92d-31dc94b80c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:50.490896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1105/2176] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:50 2022] DELETE /identity/v3/projects/4f2c3307a1e94d99a5f6426aefbdd8f8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:50.495812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:50.496277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:50.496368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:50.515338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:50.516556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:50.574469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1072/2177] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:50.582873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:50.583206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:50.583490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:50.612749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:50.613983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:50.622709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1106/2178] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:50.626959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:50.637488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac4caa-75f5-49bc-bfff-1db04f9295b3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:50.643347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:50.643703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:50.644032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:50.644310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:50.645039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:50.645375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:50.647053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:50.662070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1073/2179] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:50.667384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:50.677638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafae43-be0f-48f9-8cd0-b51fe6e4a056 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:50.682332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:50.683015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:50.683347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:50.748340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:50.748340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:50.748340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:50.931535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:50.998171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1107/2180] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:50 2022] POST /identity/v3/projects => generated 375 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:51.002351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:51.012483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f97b-b379-4c66-ab06-f6d1fe191ce0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:51.086021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:51.086814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.087812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.087812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.088937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:51.090965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:51.251711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1074/2181] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:51 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:51.252926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:51.265620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dedfcb-49af-4090-afa1-68da55ba0555 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:51.270878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:51.280239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.280239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.280239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.280239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:51.280239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:51.288659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1108/2182] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:51.292535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:51.304198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1bdf1-5b35-469b-8686-cedf5f52bc61 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:51.309062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:51.309762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.310608 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.310891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.311809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:51.313565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adc610-3ca4-4edf-8902-3190e8c5b7af admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:51.327505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1075/2183] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:51.353874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:51.365977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbbfa6-7564-4062-a222-e53024526386 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:51.387598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:51.389089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.393660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.394455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] PATH_INFO: `/v3/projects/911df09e814e4120842a0e8bc7225bb5/users/c09ff83a62594a2ba3112459bb1c168b/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.426995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911df09e814e4120842a0e8bc7225bb5, user_id=c09ff83a62594a2ba3112459bb1c168b, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:51.429730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:51.540684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1109/2184] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:51 2022] PUT /identity/v3/projects/911df09e814e4120842a0e8bc7225bb5/users/c09ff83a62594a2ba3112459bb1c168b/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:51.550858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.551358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.552034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.584629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] MFA Rules not processed for user `c09ff83a62594a2ba3112459bb1c168b`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:51.586100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:51.710958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1076/2185] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:51.720911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:51.722771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:51.805939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33200-4bbd-4ffd-b325-4cf195ca0a73 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:51.812255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:51.813032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:51.813725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:51.813725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:51.823848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:51.825590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:51.845300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1110/2186] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:52.238032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:53.163487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:53.165058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:53.317885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3966a-e215-4190-8aa6-7d6585093b3a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:53.323360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:53.324536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:53.325459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:53.325942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] PATH_INFO: `/v3/users/9d69fb173f48483f9efef0cf6627e453` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:53.328272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d69fb173f48483f9efef0cf6627e453)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:53.330814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:53.783864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d69fb173f48483f9efef0cf6627e453'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:53.784585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d69fb173f48483f9efef0cf6627e453'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:53.791962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d69fb173f48483f9efef0cf6627e453'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:54.047144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d69fb173f48483f9efef0cf6627e453'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:54.062600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0d05f354-2cfc-4783-a3e7-f8e627aac425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d69fb173f48483f9efef0cf6627e453'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:54.081792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1077/2187] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:53 2022] DELETE /identity/v3/users/9d69fb173f48483f9efef0cf6627e453 => generated 0 bytes in 919 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:54.731172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:54.748734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b15e26-a351-413b-bd15-73cc8c4904b6 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:54.856715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:54.857702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:54.858059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:54.858369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] PATH_INFO: `/v3/projects/5f5c3fd01b334902bae81d9929cfa05a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:54.859689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5c3fd01b334902bae81d9929cfa05a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:54.861911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:54.900372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:54.901264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:54.901715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:54.923345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] MFA Rules not processed for user `4df05cbf303c4524a665b429f240a17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:54.924576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:54.985713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1078/2188] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:55.094577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5c3fd01b334902bae81d9929cfa05a', 'request_id': 'req-dd3bf635-f221-4877-8d71-ce65b7212bf3'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.101830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5c3fd01b334902bae81d9929cfa05a', 'request_id': 'req-dd3bf635-f221-4877-8d71-ce65b7212bf3'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.239866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:55.240325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5c3fd01b334902bae81d9929cfa05a', 'request_id': 'req-dd3bf635-f221-4877-8d71-ce65b7212bf3'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.259922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] The token cache is being invalidate because project 5f5c3fd01b334902bae81d9929cfa05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:55.260325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.264028 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.266041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1111/2189] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:54 2022] DELETE /identity/v3/projects/5f5c3fd01b334902bae81d9929cfa05a => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:55.269810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:55.271252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:55.288242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bb8e8-d9fc-4912-b489-374ab25324ea None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:55.460116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:55.461454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:55.461894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:55.462336 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] PATH_INFO: `/v3/users/a36a8a56df3c43c290e7020109cbeb99` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:55.472546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36a8a56df3c43c290e7020109cbeb99)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:55.475219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:55.963798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36a8a56df3c43c290e7020109cbeb99'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.964897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36a8a56df3c43c290e7020109cbeb99'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.972437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36a8a56df3c43c290e7020109cbeb99'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:55.979722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:56.100182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36a8a56df3c43c290e7020109cbeb99'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.114783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-59a76ce6-3ee0-48de-a2ee-525d40725205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36a8a56df3c43c290e7020109cbeb99'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.121751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1079/2190] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:45:55 2022] DELETE /identity/v3/users/a36a8a56df3c43c290e7020109cbeb99 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:56.449262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:56.453379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:56.477449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bf635-f221-4877-8d71-ce65b7212bf3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:56.589718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:56.589718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:56.589718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:56.589718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] PATH_INFO: `/v3/projects/937fca4a7a264317a42fd6c1700b15b3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:56.590172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=937fca4a7a264317a42fd6c1700b15b3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:56.590172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:56.813620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937fca4a7a264317a42fd6c1700b15b3', 'request_id': 'req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.819593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937fca4a7a264317a42fd6c1700b15b3', 'request_id': 'req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.943504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937fca4a7a264317a42fd6c1700b15b3', 'request_id': 'req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.977112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] The token cache is being invalidate because project 937fca4a7a264317a42fd6c1700b15b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:45:56.977942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.978624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-be8f184e-a04a-49c5-bfa6-8a0cc7a79546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:45:56.984348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1112/2191] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:45:56 2022] DELETE /identity/v3/projects/937fca4a7a264317a42fd6c1700b15b3 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:56.986163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:56.987252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:56.987896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.009355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:57.010564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:57.085697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1080/2192] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:57.094816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.095098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.095343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.118162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:57.120372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:57.133337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1113/2193] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:57.138449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.149874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f45c1-e1f2-41c3-8062-c147508ba92c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.156483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.157286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.157658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.158021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.158964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:45:57.159503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.170319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.186966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1081/2194] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:45:57.190658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.202605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afee417-807e-4d0b-a4f7-72c2441a7e7d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.210982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.211918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.214601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.214601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.214601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.216790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.368750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1114/2195] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:57 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:57.374739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.388990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430623-bf17-4d7e-940b-fff1ddeadac4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.462540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.463559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.463994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.464442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.465431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.468109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.563846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1082/2196] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:57 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:57.569220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.580306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d3c3-2639-40f9-9cb4-404e67084daa admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.584932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.585678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.585995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.591526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.591526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.591526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.601317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1115/2197] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:57.605489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.616036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8e8da-23c1-4a74-b628-90b2889e430e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.630194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.630194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.630194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.630194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.630645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.630645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43a6c7-c75c-460e-aa82-8b6a98b2cf12 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.635779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1083/2198] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:57.640255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:57.653215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075164-deb6-4a72-8d61-c876e7bcbf00 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:57.658533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:57.659540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.659899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.660252 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] PATH_INFO: `/v3/projects/e7a7a3f996e6495c93f032797fc42a64/users/e224187eb92c4145a92fe659d1da8ea6/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.677935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a7a3f996e6495c93f032797fc42a64, user_id=e224187eb92c4145a92fe659d1da8ea6, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:57.680998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:57.842361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1116/2199] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:57 2022] PUT /identity/v3/projects/e7a7a3f996e6495c93f032797fc42a64/users/e224187eb92c4145a92fe659d1da8ea6/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:45:57.852542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:57.852834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:57.853116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:57.884266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] MFA Rules not processed for user `e224187eb92c4145a92fe659d1da8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:57.885398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:57.953476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1084/2200] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:57.957785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.021927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156706b0-efa8-4527-aa8e-538bf4f7ffca admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.028703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.029388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.029672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.029947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.030643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.032712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.150312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1117/2201] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:45:57 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:45:58.151734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.164717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08edacf-78c5-4cfb-a268-f09b61892167 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.226772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.227501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.227808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.228078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.228776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.230706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.366626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1085/2202] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:45:58 2022] POST /identity/v3/users => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:45:58.379509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.393012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce82931-350c-4379-8204-015806ac1d85 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.398445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.399223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.399509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.399785 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.400710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.402247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.426061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1118/2203] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:58.426165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.436041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05151ee9-2ab2-45db-9d47-35af7dff4f5e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.442439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.443746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.443746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.444206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] PATH_INFO: `/v3/projects/6870848d255a4cf58f7e2f4d76655349/users/4efc4e3fe0404b208f93638b62990a91/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.464159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6870848d255a4cf58f7e2f4d76655349, user_id=4efc4e3fe0404b208f93638b62990a91, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.468047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.545513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1086/2204] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:58 2022] PUT /identity/v3/projects/6870848d255a4cf58f7e2f4d76655349/users/4efc4e3fe0404b208f93638b62990a91/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:58.550243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.559688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cab87-ce64-4744-abe2-80080db64668 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.656448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.657716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.657716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.657716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.658338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.659988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccdc8f-9cd1-4f9a-a422-318f94c37170 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.672443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1119/2205] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:45:58.676702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.703859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999bba-0307-4342-89fd-16fb166a6f3f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.717853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.720038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.727847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.728296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] PATH_INFO: `/v3/projects/6870848d255a4cf58f7e2f4d76655349/users/4efc4e3fe0404b208f93638b62990a91/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.730633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6870848d255a4cf58f7e2f4d76655349, user_id=4efc4e3fe0404b208f93638b62990a91, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:58.735673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:58.758631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1087/2206] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:45:58 2022] PUT /identity/v3/projects/6870848d255a4cf58f7e2f4d76655349/users/4efc4e3fe0404b208f93638b62990a91/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:45:58.760904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.761435 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.761724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:58.800768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] MFA Rules not processed for user `4efc4e3fe0404b208f93638b62990a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:45:58.801994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:58.885384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1120/2207] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:45:58.899697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:45:58.901425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:45:58.962800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e50ae-fc94-4d80-88d9-79011fa40789 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:45:58.966459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:45:58.967212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:45:58.967499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:45:58.967777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:45:59.035592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:45:59.037071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:45:59.048062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1088/2208] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:45:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:45:59.548040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:45:59.968862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:01.313987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:05.524831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:05.635279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:05.635780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:05.703909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35eaa7-0aba-4e34-8e7a-82ca14384324 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:05.708617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:05.709831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:05.710150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:05.710429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] PATH_INFO: `/v3/users/c09ff83a62594a2ba3112459bb1c168b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:05.714996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09ff83a62594a2ba3112459bb1c168b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:05.717055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:05.751343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:05.752792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:05.764000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead6bae-e72b-4770-8de6-b163b7f83ef5 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:05.767777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:05.768411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:05.768692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:05.769085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:05.778182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:05.780165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff1f45-ea9e-4edb-9710-bf1310310525 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:05.789813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1089/2209] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:46:06.022180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09ff83a62594a2ba3112459bb1c168b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:06.022834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09ff83a62594a2ba3112459bb1c168b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:06.030195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09ff83a62594a2ba3112459bb1c168b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:06.153700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09ff83a62594a2ba3112459bb1c168b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:06.177133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09ff83a62594a2ba3112459bb1c168b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:06.186394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1121/2210] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:05 2022] DELETE /identity/v3/users/c09ff83a62594a2ba3112459bb1c168b => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:06.188922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571746-9a57-4e85-813d-171167503651 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:06.189215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571746-9a57-4e85-813d-171167503651 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:06.189515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571746-9a57-4e85-813d-171167503651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:06.208754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1b571746-9a57-4e85-813d-171167503651 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:06.209981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1b571746-9a57-4e85-813d-171167503651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:06.268416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1090/2211] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:06.278513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:06.280487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:06.343637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5c9a-f289-4766-a2ec-1e641f2fddb6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:06.350402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:06.351652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:06.352087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:06.352429 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:06.362824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:06.364314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f737576-d9cb-4446-a7b6-645829018d56 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:06.376023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1122/2212] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:07.161507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b571746-9a57-4e85-813d-171167503651 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:07.170025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1b571746-9a57-4e85-813d-171167503651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:07.192108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b571746-9a57-4e85-813d-171167503651 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:07.200219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:07.202285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.202759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.203237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] PATH_INFO: `/v3/projects/911df09e814e4120842a0e8bc7225bb5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.204863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=911df09e814e4120842a0e8bc7225bb5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:07.207016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:07.439948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911df09e814e4120842a0e8bc7225bb5', 'request_id': 'req-b454f19c-f2b1-49f4-b6c5-1c90de012464'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:07.445915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911df09e814e4120842a0e8bc7225bb5', 'request_id': 'req-b454f19c-f2b1-49f4-b6c5-1c90de012464'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:07.562007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911df09e814e4120842a0e8bc7225bb5', 'request_id': 'req-b454f19c-f2b1-49f4-b6c5-1c90de012464'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:07.576787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] The token cache is being invalidate because project 911df09e814e4120842a0e8bc7225bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:07.577427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:07.578397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b454f19c-f2b1-49f4-b6c5-1c90de012464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:07.585979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.586812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.587315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.588811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1091/2213] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:07 2022] DELETE /identity/v3/projects/911df09e814e4120842a0e8bc7225bb5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:07.616271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:07.617497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:07.679670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1123/2214] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:07.688923 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.689222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.689507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.708348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:07.709552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:07.718217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1092/2215] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:07.722298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:07.732111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1300a6-dfa5-4378-8841-95f5fc54f01a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:07.736967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:07.737616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.737896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.738174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.738893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:46:07.739225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:07.740808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:07.752136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1124/2216] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:46:07.756190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:07.766206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f550aa0-299f-4402-96be-b981ef6e1b2f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:07.771021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:07.771732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.772077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.772358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.773054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:07.775164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502270b8-29fd-48a6-8c11-086228e62965 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:07.895978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1093/2217] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:07 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:07.897221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:07.907390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff8f7-d2af-48fd-9125-eef2e1da052a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:07.968480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:07.969171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:07.969460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:07.969737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:07.970502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:07.972455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:08.006705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42731f4-9494-4800-a028-3d1260601738 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.007036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42731f4-9494-4800-a028-3d1260601738 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.007320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42731f4-9494-4800-a028-3d1260601738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.019048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:08.026692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b42731f4-9494-4800-a028-3d1260601738 None None] MFA Rules not processed for user `2d0d5723417d4b60bc51e47f28d09902`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:08.031349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b42731f4-9494-4800-a028-3d1260601738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:08.082525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1094/2218] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:08.107888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1125/2219] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:07 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:08.112403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42731f4-9494-4800-a028-3d1260601738 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:08.123884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42731f4-9494-4800-a028-3d1260601738 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:08.128547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:08.129207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.129498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.129886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.130627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:08.132548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:08.145352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1095/2220] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:08.158299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:08.168663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23c7a-8f56-4d20-a7db-95b8b3f134ed admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:08.177701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:08.178384 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.185245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.189751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.189751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:08.189751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b0d10-3474-4a69-9577-6b4cc14dc185 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:08.201647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1126/2221] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:08.205349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:08.216559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf6eb-36de-4987-bf21-a00db3e495c9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:08.230409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:08.232074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.232366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.232648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] PATH_INFO: `/v3/projects/2ab754ce6b844222af7ddb23f93aa648/users/88446ec114864875a88a20d634b4423a/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.258783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab754ce6b844222af7ddb23f93aa648, user_id=88446ec114864875a88a20d634b4423a, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:08.259049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:08.348960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1096/2222] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:08 2022] PUT /identity/v3/projects/2ab754ce6b844222af7ddb23f93aa648/users/88446ec114864875a88a20d634b4423a/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:08.353839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.354312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.354747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.375277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:08.376518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:08.453430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1127/2223] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:08.463814 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:08.465616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:08.541236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf83d1-5c15-4edb-8acb-8fc29433e705 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:08.545178 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:08.545891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:08.546019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:08.546391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:08.557368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:08.559012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f44b-c926-478a-9583-10609510abc2 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:08.571176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1097/2224] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:10.663934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:10.665343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:10.677675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fc4bb-d722-4cae-8bed-abfcefcd868d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:10.682243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:10.683131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:10.683436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:10.683695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] PATH_INFO: `/v3/users/5034fbc8f3dd464ea2d6ebb48457ff8e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:10.685722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5034fbc8f3dd464ea2d6ebb48457ff8e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:10.687569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:11.104208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5034fbc8f3dd464ea2d6ebb48457ff8e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:11.105217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5034fbc8f3dd464ea2d6ebb48457ff8e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:11.118440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5034fbc8f3dd464ea2d6ebb48457ff8e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:11.272556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5034fbc8f3dd464ea2d6ebb48457ff8e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:11.300223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5034fbc8f3dd464ea2d6ebb48457ff8e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:11.310974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1128/2225] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:10 2022] DELETE /identity/v3/users/5034fbc8f3dd464ea2d6ebb48457ff8e => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:11.313238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032d38-e770-4318-bb09-294e046d9abf None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:11.314049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032d38-e770-4318-bb09-294e046d9abf None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:11.314914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032d38-e770-4318-bb09-294e046d9abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:11.343195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e1032d38-e770-4318-bb09-294e046d9abf None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:11.344924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e1032d38-e770-4318-bb09-294e046d9abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:11.480153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1098/2226] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:11.489578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:11.491733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:11.566203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c020da-d9cf-4d99-bc0f-078b97b7ad5d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:11.570612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:11.571306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:11.571701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:11.571994 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:11.591227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:11.592690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:11.611161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1129/2227] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:46:12.125197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:12.126915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:12.126915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:12.156700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] MFA Rules not processed for user `61d74fdb719b40208ee408b838256a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:12.162335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:12.279618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1099/2228] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:12.362283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:12.365060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:12.392541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444c92-84e9-44fb-9e12-0e439892f203 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:12.399574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:12.401742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:12.403874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:12.404405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] PATH_INFO: `/v3/projects/bf2ab66fdf4c44bb9eec4d1875726b6d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:12.409317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2ab66fdf4c44bb9eec4d1875726b6d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:12.411473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:12.543111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:12.545264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:12.577654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d589da-044c-459f-a54c-e14c74ab5e67 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:12.587614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:12.588506 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:12.588786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:12.591558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] PATH_INFO: `/v3/users/e224187eb92c4145a92fe659d1da8ea6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:12.596270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e224187eb92c4145a92fe659d1da8ea6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:12.600076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:12.675615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2ab66fdf4c44bb9eec4d1875726b6d', 'request_id': 'req-adfa8c61-323c-483b-a6cd-71034ff32d31'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:12.681289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2ab66fdf4c44bb9eec4d1875726b6d', 'request_id': 'req-adfa8c61-323c-483b-a6cd-71034ff32d31'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:12.776607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:12.873648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2ab66fdf4c44bb9eec4d1875726b6d', 'request_id': 'req-adfa8c61-323c-483b-a6cd-71034ff32d31'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:12.900143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] The token cache is being invalidate because project bf2ab66fdf4c44bb9eec4d1875726b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:12.900143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:12.904319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-adfa8c61-323c-483b-a6cd-71034ff32d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:12.909911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1130/2229] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:12 2022] DELETE /identity/v3/projects/bf2ab66fdf4c44bb9eec4d1875726b6d => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:12.911613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24252604-9c5f-46ad-88fa-c37813c1dfda None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:12.922031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24252604-9c5f-46ad-88fa-c37813c1dfda None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:12.922031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24252604-9c5f-46ad-88fa-c37813c1dfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:12.954845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-24252604-9c5f-46ad-88fa-c37813c1dfda None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:12.955996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-24252604-9c5f-46ad-88fa-c37813c1dfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:13.035778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e224187eb92c4145a92fe659d1da8ea6'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:13.036906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e224187eb92c4145a92fe659d1da8ea6'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:13.041490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:13.061319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e224187eb92c4145a92fe659d1da8ea6'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:13.077737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1131/2230] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:13.095954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.096858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.097941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.133519 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:13.135024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:13.144417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1132/2231] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:13.148499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.158254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc41e53-d411-4d0f-8db7-0bad0ca62c44 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.163089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.163732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.164020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.164296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.165181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:46:13.165498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.167116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.184240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1133/2232] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:46:13.190397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e224187eb92c4145a92fe659d1da8ea6'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:13.193225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.204729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3c4c4-40b4-4569-a1a9-549592b32935 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.209982 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.210656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.210798 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-6d1162c1-7d03-4a50-b725-7fcf8211e473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e224187eb92c4145a92fe659d1da8ea6'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:13.210964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.211527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.212234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.214270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.226284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1100/2233] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:12 2022] DELETE /identity/v3/users/e224187eb92c4145a92fe659d1da8ea6 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:13.234035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.234347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.235016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.258896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:13.260150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:13.394810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1101/2234] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:13.406808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.409218 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:13.460267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1134/2235] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:13 2022] POST /identity/v3/projects => generated 369 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:46:13.461385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.571085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6491d25-42c6-4cc7-931b-39c0bbb98e90 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.612029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ed8bc-cfda-4348-800b-a08305d8b78c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.615875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.616557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.616856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.617139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.618021 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.620032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.695953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.696996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.697290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.697559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.717640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1135/2236] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:13 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:13.722068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.727179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.728857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.742002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1102/2237] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:46:13.751699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d07b18-66a5-49a0-831e-ad43df5e0cf9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.764395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.764395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.764395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.765043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.765892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.767955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.784042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1136/2238] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:13.785352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.805285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d5ae-2087-4a1f-a9a9-46231a750490 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.812637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.814271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.814271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.814547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.815323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.817128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247351ed-84ec-4466-9206-abe8b55287b2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:13.829868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1103/2239] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:13.834395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:13.857673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906330bc-7cda-423b-910b-74059c60ca48 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:13.863131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:13.864158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:13.864570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:13.864970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] PATH_INFO: `/v3/projects/293b9c4867ad4119a18655392748190d/users/07c8624990504c5aa7e30ead7c5da72b/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:13.895584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293b9c4867ad4119a18655392748190d, user_id=07c8624990504c5aa7e30ead7c5da72b, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:13.898552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:14.018461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1137/2240] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:13 2022] PUT /identity/v3/projects/293b9c4867ad4119a18655392748190d/users/07c8624990504c5aa7e30ead7c5da72b/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:14.032030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:14.032341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:14.032633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:14.111907 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] MFA Rules not processed for user `07c8624990504c5aa7e30ead7c5da72b`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:14.113476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:14.254262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1104/2241] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:14.267242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:14.269268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:14.350380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea67b-141e-4ee5-aaeb-40788cecd461 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:14.354149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:14.354808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:14.355147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:14.355430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:14.365909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:14.367412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:14.379665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1138/2242] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:14.496944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:14.498416 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:14.601712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798da8c-25e6-48db-82f3-e9076491385e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:14.626529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:14.626529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:14.628236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:14.628236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] PATH_INFO: `/v3/projects/e7a7a3f996e6495c93f032797fc42a64` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:14.629207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a7a3f996e6495c93f032797fc42a64)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:14.631327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:14.965743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a7a3f996e6495c93f032797fc42a64', 'request_id': 'req-a9f029fc-a51a-424a-badd-d98c30f1f7de'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:14.971691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a7a3f996e6495c93f032797fc42a64', 'request_id': 'req-a9f029fc-a51a-424a-badd-d98c30f1f7de'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.095059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a7a3f996e6495c93f032797fc42a64', 'request_id': 'req-a9f029fc-a51a-424a-badd-d98c30f1f7de'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.110289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] The token cache is being invalidate because project e7a7a3f996e6495c93f032797fc42a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:15.110690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.111475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.114092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1105/2243] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:14 2022] DELETE /identity/v3/projects/e7a7a3f996e6495c93f032797fc42a64 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:15.121311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:15.134876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216494f2-06c0-4a81-b8b7-1c6171632968 service glance] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:15.213149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:15.214203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:15.214487 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:15.214765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] PATH_INFO: `/v3/users/4efc4e3fe0404b208f93638b62990a91` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:15.216169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4efc4e3fe0404b208f93638b62990a91)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:15.218149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:15.707027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4efc4e3fe0404b208f93638b62990a91'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.712561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4efc4e3fe0404b208f93638b62990a91'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:15.750580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4efc4e3fe0404b208f93638b62990a91'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:16.007373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4efc4e3fe0404b208f93638b62990a91'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:16.041296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e77df4eb-2b99-4bf0-8520-9ad1dbecec82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4efc4e3fe0404b208f93638b62990a91'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:16.054083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1139/2244] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:15 2022] DELETE /identity/v3/users/4efc4e3fe0404b208f93638b62990a91 => generated 0 bytes in 933 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:17.032529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:17.034546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:17.048528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f029fc-a51a-424a-badd-d98c30f1f7de admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:17.154580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:17.155722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:17.156112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:17.156494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] PATH_INFO: `/v3/projects/6870848d255a4cf58f7e2f4d76655349` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:17.157870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6870848d255a4cf58f7e2f4d76655349)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:17.159965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:17.361292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6870848d255a4cf58f7e2f4d76655349', 'request_id': 'req-68579f80-3c68-49e0-a751-1ce2fdbc9284'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:17.373165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6870848d255a4cf58f7e2f4d76655349', 'request_id': 'req-68579f80-3c68-49e0-a751-1ce2fdbc9284'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:17.398958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:17.544471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6870848d255a4cf58f7e2f4d76655349', 'request_id': 'req-68579f80-3c68-49e0-a751-1ce2fdbc9284'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:17.582661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] The token cache is being invalidate because project 6870848d255a4cf58f7e2f4d76655349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:17.583327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:17.584188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68579f80-3c68-49e0-a751-1ce2fdbc9284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:17.593561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1106/2245] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:17 2022] DELETE /identity/v3/projects/6870848d255a4cf58f7e2f4d76655349 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:17.594840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:17.596336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:17.596719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:17.627054 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:17.632716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:17.788578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1140/2246] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:17.797575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:17.797865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:17.798139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:17.839199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:17.840981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:17.873654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1107/2247] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:17.877703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:17.888139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0e92-0b67-4450-ba08-448325b7a6ba None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:17.903589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:17.904757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:17.905060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:17.910348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:17.914284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:46:17.914645 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:17.932414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:17.975247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1141/2248] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:46:17.975342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:17.985185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64400fd0-5549-4d62-9f5d-932b3fbcbeca None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:18.014603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:18.018289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.018661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.019045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:18.019925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:18.023604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:18.184699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:18.184939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1108/2249] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:17 2022] POST /identity/v3/projects => generated 375 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:18.207093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa977c-9ad3-4099-b56a-e45c0c201e33 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:18.393164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:18.394643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.395125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.396077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:18.400222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:18.404639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:18.580997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:18.589124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1142/2250] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:18 2022] POST /identity/v3/users => generated 288 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:46:18.593834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642c8f-bcf5-49e7-80be-b04e23dc1307 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:18.611912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:18.612607 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.612902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.613184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:18.616076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:18.617913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:18.659856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:18.666033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1109/2251] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:18.674841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e934-54e0-4f44-890d-dd1bb70644fe admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:18.691301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:18.692013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.692305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.692590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:18.693339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:18.703237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc70af-240b-4520-8359-29cd260e46f9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:18.733465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1143/2252] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:46:18.738094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:18.757955 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0bca7-50e1-43fd-a29e-19c81783d2a4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:18.771919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:18.773742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.774214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.774650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] PATH_INFO: `/v3/projects/6133cfc492bb4cab90b281ac250387cc/users/20350abec1df49f79f78d3af3826257f/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:18.796246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6133cfc492bb4cab90b281ac250387cc, user_id=20350abec1df49f79f78d3af3826257f, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:18.802655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:18.972197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1110/2253] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:18 2022] PUT /identity/v3/projects/6133cfc492bb4cab90b281ac250387cc/users/20350abec1df49f79f78d3af3826257f/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:18.983515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:18.983805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:18.984694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:19.060666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] MFA Rules not processed for user `20350abec1df49f79f78d3af3826257f`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:19.061962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:19.136184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:19.225282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:19.241676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1144/2254] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:19.568755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266f1b2-774c-42d5-b15d-9331f316925d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:19.568755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:19.568755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:19.568755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:19.572152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:19.572152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:19.572152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8f82-a242-4c5b-832c-54c7b3639959 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:19.938262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1111/2255] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:19 2022] POST /identity/v3/projects => generated 373 bytes in 714 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:46:19.942667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:19.961029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d2b53-6947-463a-bb41-b5d18d32f5bb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.074386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.075511 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.076148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.077161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.078221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.080668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.139976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459a83-1315-428d-986a-49207f6abd40 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.139976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459a83-1315-428d-986a-49207f6abd40 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.139976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459a83-1315-428d-986a-49207f6abd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.197492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-05459a83-1315-428d-986a-49207f6abd40 None None] MFA Rules not processed for user `88446ec114864875a88a20d634b4423a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:20.200345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-05459a83-1315-428d-986a-49207f6abd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:20.311107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1145/2256] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:19 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:20.311323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1112/2257] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:20.316213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.316361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459a83-1315-428d-986a-49207f6abd40 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.317939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:20.327304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459a83-1315-428d-986a-49207f6abd40 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.331177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91962a-003c-4d6c-9aee-2afb9513f187 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.332088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.332760 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.333049 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.333330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.334068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.335710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.337696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.339035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.339547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.340635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.341547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.344230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.346067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1113/2258] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:20.363184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.367401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd140e3-1d0f-4754-be7f-6fb9988550d2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.372826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.373738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.374027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.374303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] PATH_INFO: `/v3/projects/ae2d83472fd840b988f2e630982de6a7/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.389355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2d83472fd840b988f2e630982de6a7, user_id=ea2de1bd322f4a6f9b7dc39ffa5f8ae5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.392925 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.516001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1114/2259] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:20 2022] PUT /identity/v3/projects/ae2d83472fd840b988f2e630982de6a7/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:46:20.520692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1146/2260] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:20 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:20.520875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.522107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.603999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c39c-938b-43d9-8b08-1f8d54c6a126 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.609821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.610712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.611259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.611745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.611914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23f98-24e2-489a-b102-593e68f16cfe admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.612497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.614096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.618027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.618774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.619115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.619874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.620640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.623026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.624692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1115/2261] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:20.630409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.640785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df7ee-9ac8-40d5-89d1-1b32146ff7f6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.645604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.646486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.646771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.647133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] PATH_INFO: `/v3/projects/ae2d83472fd840b988f2e630982de6a7/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.649479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2d83472fd840b988f2e630982de6a7, user_id=ea2de1bd322f4a6f9b7dc39ffa5f8ae5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.652513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fa394-0c77-4e49-a5ac-e4dcf726c6ff admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.674176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1116/2262] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:20 2022] PUT /identity/v3/projects/ae2d83472fd840b988f2e630982de6a7/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:46:20.676113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.676410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.676769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.713311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] MFA Rules not processed for user `ea2de1bd322f4a6f9b7dc39ffa5f8ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:20.714865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:20.825590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1117/2263] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:20.834712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.835854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:20.861156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1147/2264] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:20 2022] POST /identity/v3/users => generated 299 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:20.864803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.877810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f5f7-3601-4d43-8f12-608f777ce6f3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.946788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c55c-b719-4c60-a329-929a16384516 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.948925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.950150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.950444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.950725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.951123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.951507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:20.951895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.952180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.952474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:20.953110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:20.963462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1148/2265] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:20.977980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:20.988176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dedff-2f53-483f-9846-9f07d0d2e1ae admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:20.992794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:20.993681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:20.994026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:20.994289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] PATH_INFO: `/v3/projects/fb3439d6822b4cb1af0fffd7c7943716/users/c0968fcc8f324700910ab82cc4d1eefe/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:21.015303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:21.027108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3439d6822b4cb1af0fffd7c7943716, user_id=c0968fcc8f324700910ab82cc4d1eefe, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:21.027288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:21.027432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:21.027537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:21.040156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1118/2266] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:21.150060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1149/2267] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:20 2022] PUT /identity/v3/projects/fb3439d6822b4cb1af0fffd7c7943716/users/c0968fcc8f324700910ab82cc4d1eefe/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:46:21.153917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:21.175047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8774403-1c31-4fd8-b7bf-ae77ea17e934 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:21.322001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:21.322844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:21.323243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:21.323803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:21.324465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:21.326549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:21.343549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1119/2268] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:21.347352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:21.363706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba66068-d9fe-4563-9979-66f4148b6924 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:21.368798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:21.369706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:21.370001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:21.370544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] PATH_INFO: `/v3/projects/fb3439d6822b4cb1af0fffd7c7943716/users/c0968fcc8f324700910ab82cc4d1eefe/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:21.372763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3439d6822b4cb1af0fffd7c7943716, user_id=c0968fcc8f324700910ab82cc4d1eefe, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:21.376563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515baa8-f6b4-4c88-82f3-412344f6bb86 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:21.388646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1150/2269] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:21 2022] PUT /identity/v3/projects/fb3439d6822b4cb1af0fffd7c7943716/users/c0968fcc8f324700910ab82cc4d1eefe/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:46:23.231950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:24.758228 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:24.770836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd6a1-b66e-43f6-8850-b3bd97c3c5d6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:24.838569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:24.839773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:24.840175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:24.840521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] PATH_INFO: `/v3/users/20350abec1df49f79f78d3af3826257f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:24.841927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=20350abec1df49f79f78d3af3826257f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:24.844196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:25.186674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20350abec1df49f79f78d3af3826257f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:25.187409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20350abec1df49f79f78d3af3826257f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:25.194846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20350abec1df49f79f78d3af3826257f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:25.299775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20350abec1df49f79f78d3af3826257f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:25.315656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20350abec1df49f79f78d3af3826257f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:25.323304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1120/2270] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:24 2022] DELETE /identity/v3/users/20350abec1df49f79f78d3af3826257f => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:25.328073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:25.328614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:25.328614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:25.357917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:25.359179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:25.438949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1151/2271] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:25.449029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:25.451098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:25.513560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe317f0-db5b-43f6-a616-ebda96ce50cd admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:25.517417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:25.518077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:25.518377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:25.518735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:25.530969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:25.531203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:25.547009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1121/2272] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:46:26.113180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:26.767177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:26.769237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:26.797260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cba077-3048-4449-bb41-e3d54510b963 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:26.807454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:26.808606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:26.809087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:26.809366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] PATH_INFO: `/v3/projects/6133cfc492bb4cab90b281ac250387cc` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:26.811375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6133cfc492bb4cab90b281ac250387cc)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:26.813893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:27.009033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6133cfc492bb4cab90b281ac250387cc', 'request_id': 'req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.020689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6133cfc492bb4cab90b281ac250387cc', 'request_id': 'req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.123847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6133cfc492bb4cab90b281ac250387cc', 'request_id': 'req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.138556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] The token cache is being invalidate because project 6133cfc492bb4cab90b281ac250387cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:27.138854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.139949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7ff1ce22-4259-4f89-a4ae-bdab4afee97d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.142214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1152/2273] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:26 2022] DELETE /identity/v3/projects/6133cfc492bb4cab90b281ac250387cc => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:27.146269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:27.178430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdfe39-d38c-482b-b6cf-cca5f4458e32 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:27.266846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:27.267747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:27.268037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:27.268303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] PATH_INFO: `/v3/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:27.269514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2de1bd322f4a6f9b7dc39ffa5f8ae5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:27.271471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:27.545597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2de1bd322f4a6f9b7dc39ffa5f8ae5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.548811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2de1bd322f4a6f9b7dc39ffa5f8ae5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.571673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2de1bd322f4a6f9b7dc39ffa5f8ae5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.709727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2de1bd322f4a6f9b7dc39ffa5f8ae5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.728561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:27.730117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:27.731311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:27.737991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2de1bd322f4a6f9b7dc39ffa5f8ae5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:27.756319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1122/2274] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:27 2022] DELETE /identity/v3/users/ea2de1bd322f4a6f9b7dc39ffa5f8ae5 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:27.772715 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:27.774151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:27.861632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1153/2275] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:27.917106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:27.919090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:28.010829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806ed81-8116-4238-9861-42ac8f89c05b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:28.022259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:28.024459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:28.025007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:28.025602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:28.037456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:28.039439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fdf6-11d2-4e74-b1de-556b991253a1 service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:28.051794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1123/2276] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:46:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:46:28.600546 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:28.605799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:28.765851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1d89b-100f-4445-8fc9-8cde638b68e9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:28.774191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:28.796151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:28.797373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:28.797766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] PATH_INFO: `/v3/projects/ae2d83472fd840b988f2e630982de6a7` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:28.802424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2d83472fd840b988f2e630982de6a7)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:28.806806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:28.960194 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2d83472fd840b988f2e630982de6a7', 'request_id': 'req-dae0fc86-9b79-4119-8a84-c8d1d1319d54'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:28.976667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2d83472fd840b988f2e630982de6a7', 'request_id': 'req-dae0fc86-9b79-4119-8a84-c8d1d1319d54'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:29.066989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2d83472fd840b988f2e630982de6a7', 'request_id': 'req-dae0fc86-9b79-4119-8a84-c8d1d1319d54'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:29.122109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] The token cache is being invalidate because project ae2d83472fd840b988f2e630982de6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:29.122521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:29.124193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-dae0fc86-9b79-4119-8a84-c8d1d1319d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:29.130996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1154/2277] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:28 2022] DELETE /identity/v3/projects/ae2d83472fd840b988f2e630982de6a7 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:29.133053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:29.133053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:29.133654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:29.169808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:29.172998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:29.413428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1124/2278] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:29.419107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:29.419399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:29.420354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:29.467965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:29.469230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:29.532287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1155/2279] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:29.536413 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:29.563543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c69ef-6f44-4991-9ba1-f1cbb8e88181 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:29.581858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:29.583015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:29.583720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:29.584184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:29.585191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:46:29.585841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:29.587105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e026c-3ccd-4c6a-91df-2d64b9356c7b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:29.627688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:29.637803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a978a89-0729-4c29-a587-7ddf639f90d2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:29.639095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1125/2280] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:46:29.654848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:29.655482 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:29.655772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:29.656053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:29.656862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:29.658800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:29.731291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:29.731291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:29.731291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:29.765880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] MFA Rules not processed for user `c0968fcc8f324700910ab82cc4d1eefe`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:29.767300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:29.883471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1126/2281] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:29.911326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:29.913189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:29.954260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1156/2282] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:29 2022] POST /identity/v3/projects => generated 385 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:46:29.959064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:29.974382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:30.085350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705244ce-9242-4f83-8be8-49f4ec9e8e45 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:30.146092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314c280-a652-4571-bca8-1a69f44bc705 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:30.190444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:30.196824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:30.197394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:30.197844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:30.200185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:30.205056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:30.315735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:30.316612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:30.321794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:30.327380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:30.334110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:30.517727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:30.521126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:30.541485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1127/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:46:30.620996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1157/2284] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:29 2022] POST /identity/v3/users => generated 293 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:30.627086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:30.641976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf252a-f930-4d38-99db-a1a6923ff09b service glance] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:30.655205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:30.656139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:30.663803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:30.674665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:30.674665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:30.678679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:30.726345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1128/2285] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:30.729731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:30.742690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb145c1-6ac2-4c27-af44-b3548c57a4f7 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:30.772661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:30.773616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:30.774080 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:30.774525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:30.775458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:30.777270 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056a5b0-6fda-4cd4-8fa7-f44c834c1daf admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:30.814093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1158/2286] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:30.818430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:30.853036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec598-84c6-421b-b79e-8e7882bd9681 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:30.884815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:30.885998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:30.886281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:30.886622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] PATH_INFO: `/v3/projects/e5bc996d8bda4fc59322b9dcfa5ca9f2/users/1d5aa00e588845ccae4db0b2645152b4/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:30.905148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:30.928938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5bc996d8bda4fc59322b9dcfa5ca9f2, user_id=1d5aa00e588845ccae4db0b2645152b4, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:30.933724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6e010-249d-4dfc-8303-6dc98489e680 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:31.095321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1129/2287] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:30 2022] PUT /identity/v3/projects/e5bc996d8bda4fc59322b9dcfa5ca9f2/users/1d5aa00e588845ccae4db0b2645152b4/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:31.101495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d449db-9359-4211-a32d-597631712248 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:31.102638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d449db-9359-4211-a32d-597631712248 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:31.103138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d449db-9359-4211-a32d-597631712248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:31.106478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:31.106850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:31.107246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:31.136646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-05d449db-9359-4211-a32d-597631712248 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:31.138798 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-05d449db-9359-4211-a32d-597631712248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:31.189374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] MFA Rules not processed for user `1d5aa00e588845ccae4db0b2645152b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:31.194795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:31.231492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1159/2288] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:46:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:31.261284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d449db-9359-4211-a32d-597631712248 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:31.266344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-05d449db-9359-4211-a32d-597631712248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:31.313370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1130/2289] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:31.317461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:31.396090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d449db-9359-4211-a32d-597631712248 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:31.401440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:31.402118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:31.402444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:31.402727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:31.416701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:31.418879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:31.429150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1160/2290] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:46:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:46:31.487846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac052bb-3e95-49f0-8a36-7328685f3e48 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:31.498686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:31.499431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:31.504240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:31.504604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:31.505392 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:31.513442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:31.720212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1131/2291] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:31 2022] POST /identity/v3/projects => generated 383 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:31.721373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:31.752954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff6000-e830-4f41-8cf3-7042eff142f5 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:31.913146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:31.917564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:31.919733 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:31.921881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:31.923378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:31.930438 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:32.111960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1161/2292] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:31 2022] POST /identity/v3/users => generated 292 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:32.113489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:32.141497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29949e11-7c5f-470e-adf8-9c244347e375 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:32.174452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:32.175441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:32.179691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:32.181534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:32.188085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:32.195075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:32.228719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1132/2293] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:46:32.233470 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:32.250591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851ea0-c4ec-4a1f-81ae-1a2c8044b324 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:32.289098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:32.289759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:32.290046 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:32.290560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:32.291514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:32.293387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bf793-7422-4569-bd0e-25cb6275b726 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:32.335304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1162/2294] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:32.339564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:32.357595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730900d5-ed4d-4ab5-8799-99e57ce814fc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:32.375381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:32.376284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:32.376572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:32.376855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] PATH_INFO: `/v3/projects/249c9c7a7f984c1e95d9e7f1c2a72884/users/d29d1beeb53445b48025f0187e30df6b/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:32.421394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249c9c7a7f984c1e95d9e7f1c2a72884, user_id=d29d1beeb53445b48025f0187e30df6b, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:32.424132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:32.429403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:32.584750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1133/2295] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:32 2022] PUT /identity/v3/projects/249c9c7a7f984c1e95d9e7f1c2a72884/users/d29d1beeb53445b48025f0187e30df6b/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:32.595539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:32.595838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:32.596174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:32.686702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] MFA Rules not processed for user `d29d1beeb53445b48025f0187e30df6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:32.692023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:32.899502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1163/2296] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:32.899701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:33.082251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3ee1a-fffe-4471-b295-134fe749fc11 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:33.092562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:33.093386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:33.093678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:33.094003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:33.094726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:33.097669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:33.272735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1134/2297] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:46:32 2022] POST /identity/v3/projects => generated 383 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:46:33.277091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:33.301764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe8210-4dbf-4d66-a5ff-7c55bcb11755 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:33.590797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:33.595090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:33.609253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:33.609253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:33.609253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:33.609253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:33.728666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1164/2298] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:46:33 2022] POST /identity/v3/users => generated 298 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:46:33.730903 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:33.742329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4ab0-856c-4813-8104-fa2b418846c2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:33.776677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:33.777962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:33.778267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:33.778601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:33.779383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:33.781212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:33.815072 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1135/2299] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:46:33.819016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:33.839082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51055e-4091-4fcd-9555-a6bc4c90554b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:33.861172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:33.862267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:33.863308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:33.863721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] PATH_INFO: `/v3/projects/87e62c1e85b5440383652c00118773af/users/b7c1f283e9d745dc9cf62b02b0506756/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:33.932621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e62c1e85b5440383652c00118773af, user_id=b7c1f283e9d745dc9cf62b02b0506756, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:33.935294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:34.065443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1165/2300] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:33 2022] PUT /identity/v3/projects/87e62c1e85b5440383652c00118773af/users/b7c1f283e9d745dc9cf62b02b0506756/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:34.071656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:34.084817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e3a2-f63d-49c1-b7cb-a75b3f444c34 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:34.279225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:34.280324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:34.280826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:34.282732 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:34.285121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:34.287959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e72f5-1800-460d-a696-bb69e7887df3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:34.308478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1136/2301] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:46:34.324866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:34.346979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc2c1b-968e-4f77-b6bd-a83acd7561bb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:34.368348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:34.370489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:34.370489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:34.370489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] PATH_INFO: `/v3/projects/87e62c1e85b5440383652c00118773af/users/b7c1f283e9d745dc9cf62b02b0506756/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:34.373549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e62c1e85b5440383652c00118773af, user_id=b7c1f283e9d745dc9cf62b02b0506756, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:34.386685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:34.415643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1166/2302] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:46:34 2022] PUT /identity/v3/projects/87e62c1e85b5440383652c00118773af/users/b7c1f283e9d745dc9cf62b02b0506756/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:34.429701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:34.430037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:34.430396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:34.474082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] MFA Rules not processed for user `b7c1f283e9d745dc9cf62b02b0506756`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:34.475398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:34.585078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1137/2303] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:34.593591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:34.595935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:34.695947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d70415-a0bb-44e9-bd90-7aafb9ecf913 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:34.710464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:34.711296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:34.711791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:34.712373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:34.828340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:34.829851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:34.875218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1167/2304] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:35.600051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:35.865862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:35.867672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:36.146617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c0c7-4537-4b9a-989f-d1f0f31f68e4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:36.170279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:36.171123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:36.171489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:36.174387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:36.176720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:36.224006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:36.243449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:36.298561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1138/2305] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:37.100597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:38.632481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:40.650865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:40.652827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:40.689828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7e3b5-254d-4df4-8c9c-a48461ce8be2 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:40.706425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:40.707140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:40.707672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:40.707778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:40.890097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:40.891879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc846-e354-422d-9168-ffd12a2bf193 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:40.925999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1168/2306] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:41.492571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:43.094956 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:45.620078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:45.634293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f04ce-1bd3-4d47-8f76-26496fea686e service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:45.638437 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:45.639310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:45.643215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:45.643680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:45.680076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:45.681557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:45.703521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1139/2307] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:47.221261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:50.500013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:50.787105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c77ce-5a4a-4cb4-8290-b89d0561b4fb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:50.787646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c77ce-5a4a-4cb4-8290-b89d0561b4fb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:50.787760 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c77ce-5a4a-4cb4-8290-b89d0561b4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:50.805481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-2f4c77ce-5a4a-4cb4-8290-b89d0561b4fb None None] MFA Rules not processed for user `88446ec114864875a88a20d634b4423a`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:50.806693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2f4c77ce-5a4a-4cb4-8290-b89d0561b4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:50.860605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1169/2308] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:53.454142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:54.854383 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:54.916129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba085f41-61e8-47c8-bd8b-65383add831b service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:54.921248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:54.922134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:54.922419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:54.922694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] PATH_INFO: `/v3/users/1d5aa00e588845ccae4db0b2645152b4` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:54.923937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d5aa00e588845ccae4db0b2645152b4)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:54.925862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:55.279499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d5aa00e588845ccae4db0b2645152b4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:55.280163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d5aa00e588845ccae4db0b2645152b4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:55.288859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:55.291912 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d5aa00e588845ccae4db0b2645152b4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:55.499270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d5aa00e588845ccae4db0b2645152b4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:55.513000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d5aa00e588845ccae4db0b2645152b4'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:55.519254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1140/2309] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:54 2022] DELETE /identity/v3/users/1d5aa00e588845ccae4db0b2645152b4 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:55.523684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:55.524145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:55.524239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:55.542512 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:55.543813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:55.606503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1170/2310] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:46:55.616174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:55.617575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:55.674248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9843232-6985-48fd-b6b7-4f00c8c08e80 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:55.677778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:55.678412 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:55.678696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:55.678996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:55.688015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:55.689433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:55.699357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1141/2311] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:46:55.711132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:56.281641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:56.283220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:56.296073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7dbe2-59f6-4719-b427-f101f92e13b3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:56.300706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:56.301569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:56.301884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:56.302161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] PATH_INFO: `/v3/projects/e5bc996d8bda4fc59322b9dcfa5ca9f2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:56.303408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5bc996d8bda4fc59322b9dcfa5ca9f2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:56.305326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:56.501388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5bc996d8bda4fc59322b9dcfa5ca9f2', 'request_id': 'req-44c2539b-6364-4232-8c22-ed9784a42d00'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:56.506901 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5bc996d8bda4fc59322b9dcfa5ca9f2', 'request_id': 'req-44c2539b-6364-4232-8c22-ed9784a42d00'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:56.725753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5bc996d8bda4fc59322b9dcfa5ca9f2', 'request_id': 'req-44c2539b-6364-4232-8c22-ed9784a42d00'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:56.738697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] The token cache is being invalidate because project e5bc996d8bda4fc59322b9dcfa5ca9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:56.739077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:56.739857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:56.741337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1171/2312] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:56 2022] DELETE /identity/v3/projects/e5bc996d8bda4fc59322b9dcfa5ca9f2 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:56.745583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:56.755522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74f7-9e34-4b05-8da3-276ac879bc8f service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:56.810311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:56.811201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:56.811484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:56.811843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] PATH_INFO: `/v3/users/d29d1beeb53445b48025f0187e30df6b` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:56.813029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29d1beeb53445b48025f0187e30df6b)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:56.814951 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:57.083177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29d1beeb53445b48025f0187e30df6b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:57.083899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29d1beeb53445b48025f0187e30df6b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:57.091745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29d1beeb53445b48025f0187e30df6b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:57.225090 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29d1beeb53445b48025f0187e30df6b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:57.238528 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29d1beeb53445b48025f0187e30df6b'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:57.244573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1142/2313] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:56 2022] DELETE /identity/v3/users/d29d1beeb53445b48025f0187e30df6b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:57.400505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:57.803455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:57.805074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:57.819094 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2539b-6364-4232-8c22-ed9784a42d00 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:57.880933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:57.880933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:57.880933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:57.880933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] PATH_INFO: `/v3/projects/249c9c7a7f984c1e95d9e7f1c2a72884` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:57.881460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249c9c7a7f984c1e95d9e7f1c2a72884)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:57.883358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:58.083837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249c9c7a7f984c1e95d9e7f1c2a72884', 'request_id': 'req-1012b780-0106-4d41-949a-5c5e49074385'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.090241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249c9c7a7f984c1e95d9e7f1c2a72884', 'request_id': 'req-1012b780-0106-4d41-949a-5c5e49074385'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.213359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249c9c7a7f984c1e95d9e7f1c2a72884', 'request_id': 'req-1012b780-0106-4d41-949a-5c5e49074385'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.234820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] The token cache is being invalidate because project 249c9c7a7f984c1e95d9e7f1c2a72884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:46:58.234820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.234820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1012b780-0106-4d41-949a-5c5e49074385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.234820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1172/2314] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:57 2022] DELETE /identity/v3/projects/249c9c7a7f984c1e95d9e7f1c2a72884 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:58.235131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:58.235131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:58.247083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96eb7c-02d5-4779-aff7-0838a804c2cc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:58.304204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:58.305087 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:58.305371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:58.305665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] PATH_INFO: `/v3/users/b7c1f283e9d745dc9cf62b02b0506756` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:58.306860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c1f283e9d745dc9cf62b02b0506756)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:58.309358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:58.753284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c1f283e9d745dc9cf62b02b0506756'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.754005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c1f283e9d745dc9cf62b02b0506756'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.764322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c1f283e9d745dc9cf62b02b0506756'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.874454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c1f283e9d745dc9cf62b02b0506756'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.888781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c1f283e9d745dc9cf62b02b0506756'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:46:58.895311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1143/2315] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:46:58 2022] DELETE /identity/v3/users/b7c1f283e9d745dc9cf62b02b0506756 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:46:59.111680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:59.113014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:59.113312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:59.167118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:46:59.168366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:59.242100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1173/2316] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:46:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:46:59.254664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:59.256138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:59.373605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac63b-4d64-41fa-808c-c46672369d76 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:59.376792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:59.377846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:59.380522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:59.380819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:59.393843 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:59.396204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb5fc1-48d5-4c32-9374-4f45b4a6e8ff service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:46:59.401883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:46:59.406231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1144/2317] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:46:59.722623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:46:59.726995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:46:59.853161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d803bf-34de-4d48-bb83-187cf24c43eb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:46:59.858383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:46:59.859433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:46:59.859903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:46:59.860345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] PATH_INFO: `/v3/projects/87e62c1e85b5440383652c00118773af` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:46:59.861792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e62c1e85b5440383652c00118773af)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:46:59.864073 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:00.187164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e62c1e85b5440383652c00118773af', 'request_id': 'req-0902e153-892a-448f-b008-bcc2faf0362b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:00.195986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e62c1e85b5440383652c00118773af', 'request_id': 'req-0902e153-892a-448f-b008-bcc2faf0362b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:00.259839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:00.368372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e62c1e85b5440383652c00118773af', 'request_id': 'req-0902e153-892a-448f-b008-bcc2faf0362b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:00.382995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] The token cache is being invalidate because project 87e62c1e85b5440383652c00118773af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:47:00.383289 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:00.383947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0902e153-892a-448f-b008-bcc2faf0362b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:00.385864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1174/2318] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:46:59 2022] DELETE /identity/v3/projects/87e62c1e85b5440383652c00118773af => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:00.390762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:00.391093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:00.391375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:00.415051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:00.416816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:00.597792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1145/2319] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:00.665180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:00.665872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:00.667752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:00.688494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:00.689979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:00.705152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:00.714271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1175/2320] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:00.723579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aa305-1e76-485c-9dd5-d47abb33d45a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:00.731852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:00.735623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:00.735623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:00.735623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:00.735623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:47:00.735623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:00.736602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce4d8c-fe13-48df-bc9e-73cd362de8d2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:00.754326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1146/2321] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:47:00.759620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:00.770073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:00.772606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:00.775401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:00.787333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c69a7-597b-4e98-a8fb-d959ff06bf64 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:00.796817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:00.797868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:00.798369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:00.799099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:00.800296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:00.803125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:00.829683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:00.840004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:00.913521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1176/2322] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:47:00 2022] POST /identity/v3/projects => generated 397 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:47:00.916953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:00.964700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1147/2323] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:01.054476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:01.056519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:01.061820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6120df-868b-4d72-87ad-c02303fb610a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:01.077311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:01.084600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:01.086167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:01.086464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:01.088365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:01.153669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:01.206395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5ed75-fd5f-4096-8684-a390ce3778c2 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:01.277268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:01.277990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:01.278292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:01.278481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:01.358888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:01.363846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:01.376911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1148/2324] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:47:01.804908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1177/2325] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:47:00 2022] POST /identity/v3/users => generated 299 bytes in 886 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:47:01.805502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:01.822616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146133c-48f1-48c3-a71f-a55768ca5ee8 service swift] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:01.829215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:01.830159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:01.830669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:01.831145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:01.832299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:01.834480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:01.854822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:01.855936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1149/2326] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:47:01.878857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09c78-bc9b-4c99-9ace-362c5f45eacb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:01.890999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:01.893227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:01.894035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:01.894630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:01.896173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:01.903514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4521f2-8593-4b4b-b736-47ada8a5fc54 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:01.933594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:01.936354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1178/2327] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:47:01.960152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926326df-a1b3-4638-9d46-82f2233a7690 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:01.983161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:01.983161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:01.983161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:01.983161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] PATH_INFO: `/v3/projects/849f44ed55ea49be980cdc6e0789a202/users/d5b7d1341cac4f24b40becbd1306bba1/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:02.043431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849f44ed55ea49be980cdc6e0789a202, user_id=d5b7d1341cac4f24b40becbd1306bba1, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:02.052175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:02.531633 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1150/2328] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:47:01 2022] PUT /identity/v3/projects/849f44ed55ea49be980cdc6e0789a202/users/d5b7d1341cac4f24b40becbd1306bba1/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:02.547867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330593f1-7736-4040-9a67-342be4f7cc98 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:02.548888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330593f1-7736-4040-9a67-342be4f7cc98 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:02.549474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330593f1-7736-4040-9a67-342be4f7cc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:02.605791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-330593f1-7736-4040-9a67-342be4f7cc98 None None] MFA Rules not processed for user `d5b7d1341cac4f24b40becbd1306bba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:02.608091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-330593f1-7736-4040-9a67-342be4f7cc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:02.849412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1179/2329] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:02.862875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:02.868472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:02.983625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf3e9-53e9-4ef5-81b0-00467b2836f7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:02.993394 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:03.000578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:03.001988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:03.007023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:03.056022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:03.057641 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e28527-6077-4377-9ac1-688125665999 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:03.082250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1151/2330] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:03.398870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:03.399879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:03.402663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:03.419472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1180/2331] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:47:03 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:47:03.437434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f697-00e1-4900-a27f-a4a5a777abad None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:03.438497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f697-00e1-4900-a27f-a4a5a777abad None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:03.438497 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f697-00e1-4900-a27f-a4a5a777abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:03.561130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e2c5f697-00e1-4900-a27f-a4a5a777abad None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:03.563261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e2c5f697-00e1-4900-a27f-a4a5a777abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:03.801208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1152/2332] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:47:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:04.606834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:05.663088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:07.950959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:08.499504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:09.658088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:12.453934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:13.893295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:14.224595 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:15.293049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:15.295468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:15.414208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5584-a2bb-40ba-84ed-b854c8e1c072 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:15.418527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:15.419408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:15.419877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:15.420234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:15.436221 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:15.437747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8244982-6645-4a21-b429-a92fcc5b0822 service glance] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:15.473634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1181/2333] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:15.880742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:15.880742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:15.880742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:15.883629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1153/2334] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:47:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:47:15.891383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e640c8-d634-4159-ac2c-03669fe548d2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:15.891680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e640c8-d634-4159-ac2c-03669fe548d2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:15.891964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e640c8-d634-4159-ac2c-03669fe548d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:15.916931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-04e640c8-d634-4159-ac2c-03669fe548d2 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:15.918951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-04e640c8-d634-4159-ac2c-03669fe548d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:16.010710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1182/2335] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:47:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:16.018006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:16.019810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:16.050381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d69af0-e6e4-4f91-acab-84ca739a391f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:16.064473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:16.064473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:16.064473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:16.064473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:16.080150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:16.082504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409dc39-197f-4371-b18e-6f216f88305c service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:16.103316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1154/2336] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:16.274999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:16.274999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:16.274999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:16.274999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1183/2337] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:47:16.291710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:16.292009 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:16.292313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:16.339433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:16.340684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:16.350896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1155/2338] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:16.373956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:16.377045 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:16.394819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6d1a4-5c50-48e1-b901-5312095bb901 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:16.409691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:16.410387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:16.410673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:16.410967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:16.430621 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:16.433914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f2c41-10d8-43a3-a0e2-64b68923a6ae service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:16.457778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1184/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:18.173047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:18.441540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:19.287723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:19.899231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:20.131253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:21.357905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:21.422333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad7860-75e1-42cc-90b9-6b7fc04209f3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:21.426857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:21.427840 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:21.428122 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:21.428398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] PATH_INFO: `/v3/users/d5b7d1341cac4f24b40becbd1306bba1` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:21.429601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b7d1341cac4f24b40becbd1306bba1)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:21.432060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:21.973927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b7d1341cac4f24b40becbd1306bba1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:21.974504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b7d1341cac4f24b40becbd1306bba1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:21.981088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b7d1341cac4f24b40becbd1306bba1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:22.076370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b7d1341cac4f24b40becbd1306bba1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:22.080527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:22.092611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b7d1341cac4f24b40becbd1306bba1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:22.111190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1156/2340] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:47:21 2022] DELETE /identity/v3/users/d5b7d1341cac4f24b40becbd1306bba1 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:22.121888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:22.121888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:22.122518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:22.171713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:22.173153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:22.244636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1185/2341] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:22.255504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:22.257174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:22.317352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a86b5-8a1f-47b6-ac7b-cb06e74453d1 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:22.321337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:22.322488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:22.322788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:22.323097 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:22.332227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:22.333650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3f6f9-49c1-4a43-8877-2f85bf3ce411 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:22.343117 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1157/2342] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:22.584440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:22.866009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:22.872856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:22.879997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefddd47-8302-431f-96ba-1bd3c8a61b44 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:22.886125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:22.887909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:22.888390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:22.888957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] PATH_INFO: `/v3/projects/849f44ed55ea49be980cdc6e0789a202` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:22.892183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849f44ed55ea49be980cdc6e0789a202)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:22.894709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:23.074352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849f44ed55ea49be980cdc6e0789a202', 'request_id': 'req-c65af66d-f9a5-4ecb-804d-214128162396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:23.080865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849f44ed55ea49be980cdc6e0789a202', 'request_id': 'req-c65af66d-f9a5-4ecb-804d-214128162396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:23.195750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849f44ed55ea49be980cdc6e0789a202', 'request_id': 'req-c65af66d-f9a5-4ecb-804d-214128162396'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:23.208773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] The token cache is being invalidate because project 849f44ed55ea49be980cdc6e0789a202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:47:23.209143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:23.209856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:23.211929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1186/2343] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:47:22 2022] DELETE /identity/v3/projects/849f44ed55ea49be980cdc6e0789a202 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:23.690446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:23.690893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:23.691029 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:23.709830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:23.711112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:23.776018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1158/2344] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:47:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:47:23.796034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:23.797649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:23.868315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65af66d-f9a5-4ecb-804d-214128162396 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:23.874025 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:23.877417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:23.877417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:23.877417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:23.883782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:23.885589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:23.893789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1187/2345] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:47:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:47:24.750226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:25.510091 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:26.059036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:26.298222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:27.669427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:28.334227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:29.436351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:29.437813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:29.511414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49826b0a-1343-4330-ac3c-bb9e5f0944dc None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:29.516365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:29.517253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:29.517537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:29.517812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] PATH_INFO: `/v3/users/61d74fdb719b40208ee408b838256a0f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:29.519086 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d74fdb719b40208ee408b838256a0f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:29.530746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:29.625856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:29.628018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:29.643619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe092c6-8518-4bf5-8926-b4415d7fbf38 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:29.648141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:29.649011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:29.649300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:29.649577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] PATH_INFO: `/v3/users/07c8624990504c5aa7e30ead7c5da72b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:29.651321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c8624990504c5aa7e30ead7c5da72b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:29.653405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:29.927635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d74fdb719b40208ee408b838256a0f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:29.929591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d74fdb719b40208ee408b838256a0f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:29.938282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d74fdb719b40208ee408b838256a0f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.028983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c8624990504c5aa7e30ead7c5da72b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.029954 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c8624990504c5aa7e30ead7c5da72b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.040646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c8624990504c5aa7e30ead7c5da72b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.121795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d74fdb719b40208ee408b838256a0f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.143954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b6ed0f05-9617-45ee-91b2-f06689a61a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d74fdb719b40208ee408b838256a0f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.160175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1159/2346] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:47:29 2022] DELETE /identity/v3/users/61d74fdb719b40208ee408b838256a0f => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:47:30.162114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:30.162610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:30.163142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:30.194879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:30.196047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:30.232041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:30.254579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1160/2347] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:47:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:30.269344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c8624990504c5aa7e30ead7c5da72b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.282219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:30.283750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93f6e372-cb4d-4820-8274-c3215931a1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c8624990504c5aa7e30ead7c5da72b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:30.290005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1188/2348] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:47:29 2022] DELETE /identity/v3/users/07c8624990504c5aa7e30ead7c5da72b => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:47:30.292517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:30.294856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:30.295224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:30.295445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:30.340897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:30.343843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:30.409884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdadeac-e13a-41a8-8d5e-21c9f66cd05a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:30.433163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1189/2349] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:47:30.442991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:30.444819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:30.478219 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:30.479024 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:30.479382 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:30.479931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:30.508031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47244b-8fa9-4b9f-807e-d54e6cc38f7b None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:30.513355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:30.513601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:30.513966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:30.514099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:30.535474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:30.544294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:30.585296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1190/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:30.587739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:30.593884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:30.603837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:30.608444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119b82e-2e1a-4aba-8ac4-38f9901af723 service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:30.626821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1161/2351] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:47:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:47:30.655405 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdb53b-4188-4ff5-b0c5-0b0b4878fb3e service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:30.662420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:30.663394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:30.664097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:30.664400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] PATH_INFO: `/v3/projects/7ca93a40aa85448a8e5165f4c268678a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:30.671385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca93a40aa85448a8e5165f4c268678a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:30.674253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.015625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca93a40aa85448a8e5165f4c268678a', 'request_id': 'req-7f9fff64-0e4a-4e22-a889-59aab3cb8739'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.024032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca93a40aa85448a8e5165f4c268678a', 'request_id': 'req-7f9fff64-0e4a-4e22-a889-59aab3cb8739'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.227687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca93a40aa85448a8e5165f4c268678a', 'request_id': 'req-7f9fff64-0e4a-4e22-a889-59aab3cb8739'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.260532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] The token cache is being invalidate because project 7ca93a40aa85448a8e5165f4c268678a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:47:31.261261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.263422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7f9fff64-0e4a-4e22-a889-59aab3cb8739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.271178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1191/2352] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:47:30 2022] DELETE /identity/v3/projects/7ca93a40aa85448a8e5165f4c268678a => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:31.272845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.273150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.273467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.305719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:31.310941 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:31.406391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1162/2353] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:31.414083 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.414382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.414672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.420669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.426012 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:31.453898 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda53f-43c0-4ef0-9e30-6e835b916f7b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.459331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.460366 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.460654 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.460935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] PATH_INFO: `/v3/projects/293b9c4867ad4119a18655392748190d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.461780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:31.462377 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293b9c4867ad4119a18655392748190d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.463801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:31.464747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.482164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1192/2354] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:31.486539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.506866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8acf-5ed8-4fb6-ab19-2e49d78d3d54 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.512260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.512936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.513219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.513493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.514225 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:47:31.514535 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.516633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.532036 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1193/2355] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:47:31.535821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.550335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3c8ed-2d6e-4b3e-827a-e3dc0fcfdd64 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.557011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.557691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.557984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.558262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.559074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.560998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.671307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293b9c4867ad4119a18655392748190d', 'request_id': 'req-c2e8619a-cf0a-49ab-a298-490c291040d8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.677529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293b9c4867ad4119a18655392748190d', 'request_id': 'req-c2e8619a-cf0a-49ab-a298-490c291040d8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.707702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1194/2356] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:47:31.711614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.778200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293b9c4867ad4119a18655392748190d', 'request_id': 'req-c2e8619a-cf0a-49ab-a298-490c291040d8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.790085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04256-7a11-4219-91bd-c350f80c288e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.790845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] The token cache is being invalidate because project 293b9c4867ad4119a18655392748190d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:47:31.791273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.792037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c2e8619a-cf0a-49ab-a298-490c291040d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:47:31.797365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1163/2357] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:47:31 2022] DELETE /identity/v3/projects/293b9c4867ad4119a18655392748190d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:47:31.797560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.797560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.797560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.797560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.797933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.798933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c03f49-e433-4ecd-8957-c679f701d4a2 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.799257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c03f49-e433-4ecd-8957-c679f701d4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.799534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c03f49-e433-4ecd-8957-c679f701d4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.799656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.821344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-52c03f49-e433-4ecd-8957-c679f701d4a2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:31.838972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-52c03f49-e433-4ecd-8957-c679f701d4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:31.838972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1164/2358] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:31.841287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.841577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.841867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.860900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:31.862131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:31.870692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1165/2359] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:31.887703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.899002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3d78f-9a8b-4ac3-938e-3248459fcbf3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.905799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.905799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.905799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.906131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.906834 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:47:31.907253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.908943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:31.927975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1166/2360] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:47:31.929283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:31.940127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd34e-a220-473c-81bc-129f8d1758e7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:31.947365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:31.948172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:31.948462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:31.948740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:31.949482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:31.951645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.034502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1195/2361] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/users => generated 287 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:47:32.039048 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.040685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:32.053855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3cf28-6761-4391-b9d4-d60158a711de admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.059542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.060306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.060662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.061007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.062186 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.064070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.080943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1196/2362] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:47:32.083272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.093897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139050a3-ddb1-4a56-9083-c4d550d50969 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.098955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.100115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.105286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.106128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.107295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.109292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.122909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1197/2363] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:47:32.127442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.137844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76976f-28c1-49c5-a57f-d7b5e540ef27 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.143180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.144100 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.144728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.145020 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] PATH_INFO: `/v3/projects/d2d87fe212654666a76619c9afb14629/users/519ddd36b0084a42b18a197b0e5df24f/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.157910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1167/2364] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:47:31 2022] POST /identity/v3/projects => generated 371 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:47:32.167868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.178354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d87fe212654666a76619c9afb14629, user_id=519ddd36b0084a42b18a197b0e5df24f, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.183444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba3fc9-b8ef-4a0c-866b-b641f70575d0 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.258139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3334-08ca-422f-8a6e-63c3b06b3772 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.263747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.264504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.265051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.265349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.266531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.268308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.349644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1198/2365] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:47:32 2022] PUT /identity/v3/projects/d2d87fe212654666a76619c9afb14629/users/519ddd36b0084a42b18a197b0e5df24f/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:47:32.360458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.360721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.361007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.392483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] MFA Rules not processed for user `519ddd36b0084a42b18a197b0e5df24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:32.393769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:32.413124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1168/2366] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:47:32 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:47:32.418867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.446082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916ce9-9c18-4fdf-ad96-708b5520397e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.532449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.533329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.533963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.544925 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1199/2367] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:32.557962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.557962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.557962 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.558241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.558241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:32.569850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1169/2368] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:47:32.571052 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.584510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dec57-efd8-4785-ad5a-8d909e95d4a4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.591999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.592778 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.593296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.593661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.594473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.598159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:32.604080 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.624343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1170/2369] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:47:32.628277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.639477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a26c4-fb5b-4e1a-9e68-7ac812605b53 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.644849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.645666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017967a-1b53-4801-b4aa-2d73c8e5738e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.649212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.649212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.649212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] PATH_INFO: `/v3/projects/5fab10dc289b4b709ae82d884a355800/users/cd5957c59336440c927865c4b431f6bd/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.649464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.650568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.650860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.651182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.660142 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.661575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45573cdc-ecc6-4824-bb68-97509c5b5a03 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.663790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fab10dc289b4b709ae82d884a355800, user_id=cd5957c59336440c927865c4b431f6bd, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.666592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.670950 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1200/2370] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:47:32.773084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1171/2371] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:47:32 2022] PUT /identity/v3/projects/5fab10dc289b4b709ae82d884a355800/users/cd5957c59336440c927865c4b431f6bd/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:47:32.783897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fac4fd-b28d-4cf8-97b2-2f87cc34b420 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.784766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fac4fd-b28d-4cf8-97b2-2f87cc34b420 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.785217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fac4fd-b28d-4cf8-97b2-2f87cc34b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.817058 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d6fac4fd-b28d-4cf8-97b2-2f87cc34b420 None None] MFA Rules not processed for user `cd5957c59336440c927865c4b431f6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:32.818271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d6fac4fd-b28d-4cf8-97b2-2f87cc34b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:32.884884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1201/2372] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:32.896935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:47:32.898767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:32.969520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45aec-e651-4fa1-a182-f8987c7497ed admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:47:32.973220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:47:32.973916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:32.974205 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:32.974516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:32.983875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:47:32.985335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b16ea8-e90a-4faf-b6e0-492b80205e22 service glance] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:47:32.995583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1172/2373] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:47:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:47:33.519622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:34.894202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:34.894853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:34.894853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:34.914132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] MFA Rules not processed for user `a7fd310fe0254401b10f6b1517994233`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:34.916222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:35.026942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1202/2374] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:36.057042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:47:36.057682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:47:36.057682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:47:36.096125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] MFA Rules not processed for user `519ddd36b0084a42b18a197b0e5df24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:47:36.101543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:47:36.299478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1173/2375] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:47:38.034280 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:39.023522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:39.273501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:41.035293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=100 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:41.613113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:42.175663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:45.405536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:46.457198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:52.442105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:54.268661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:56.830890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:47:58.296751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:01.646081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=86 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:03.258884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:03.261042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:03.321233 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c18e2e-0962-4057-a207-0b173d2fb11f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:03.324854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:03.325493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:03.325778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:03.326232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:03.328737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:03.396668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:03.398088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9477f-1a71-4b31-9a93-3b2dc5961588 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:03.407505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1203/2376] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:06.410027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:06.475040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d4c4c-c226-426c-b06b-0b9cbb6a6094 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:06.480617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:06.481549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:06.481838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:06.482118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] PATH_INFO: `/v3/users/cd5957c59336440c927865c4b431f6bd` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:06.483462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5957c59336440c927865c4b431f6bd)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:06.485419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:06.836967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5957c59336440c927865c4b431f6bd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:06.837835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5957c59336440c927865c4b431f6bd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:06.845015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5957c59336440c927865c4b431f6bd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:06.980717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5957c59336440c927865c4b431f6bd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:06.989833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:06.995492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5957c59336440c927865c4b431f6bd'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:07.005035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1174/2377] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:48:06 2022] DELETE /identity/v3/users/cd5957c59336440c927865c4b431f6bd => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:48:07.006499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:07.006999 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:07.007099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:07.026591 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:07.027850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:07.137065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1204/2378] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:07.155238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:07.156283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:07.282232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f8858-3117-4429-b07b-97711c89faac admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:07.286482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:07.287242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:07.287539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:07.287821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:07.297552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:07.299730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bbc19-d4fe-4def-93fb-9bc7a5d5e9d1 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:07.312532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1175/2379] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:07.967234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:07.968808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:07.981275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4942-16fe-46de-80de-ea022c6ca7a8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:07.985949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:07.986824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:07.987180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:07.987468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] PATH_INFO: `/v3/projects/5fab10dc289b4b709ae82d884a355800` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:07.988718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fab10dc289b4b709ae82d884a355800)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:07.990652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:08.195105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:08.198422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fab10dc289b4b709ae82d884a355800', 'request_id': 'req-58cb0398-3278-4d28-8fde-5c385f0ad012'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:08.203722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fab10dc289b4b709ae82d884a355800', 'request_id': 'req-58cb0398-3278-4d28-8fde-5c385f0ad012'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:08.353668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fab10dc289b4b709ae82d884a355800', 'request_id': 'req-58cb0398-3278-4d28-8fde-5c385f0ad012'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:08.365722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] The token cache is being invalidate because project 5fab10dc289b4b709ae82d884a355800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:48:08.366033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:08.366674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:08.368893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1205/2380] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:48:07 2022] DELETE /identity/v3/projects/5fab10dc289b4b709ae82d884a355800 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:48:17.843566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=30,keystone:DELETE=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:19.082905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:19.484372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ddda3-9d4f-4bcb-a075-e1ba4b556858 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:19.484889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ddda3-9d4f-4bcb-a075-e1ba4b556858 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:19.485691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ddda3-9d4f-4bcb-a075-e1ba4b556858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:19.504441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-180ddda3-9d4f-4bcb-a075-e1ba4b556858 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:19.506395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-180ddda3-9d4f-4bcb-a075-e1ba4b556858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:19.562476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1176/2381] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:48:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:19.575394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:19.576852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:19.662424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb0398-3278-4d28-8fde-5c385f0ad012 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:19.667642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:19.668296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:19.668584 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:19.668863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:19.676955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:19.678526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:19.686762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1206/2382] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:48:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:48:25.377463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:25.378037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:25.378037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:25.395924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] MFA Rules not processed for user `2d0d5723417d4b60bc51e47f28d09902`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:25.397127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:25.486816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1177/2383] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:29.683339 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=32 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:32.624128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:32.626258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:32.690488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e28bff-d561-43e9-bbfd-3932deb05465 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:32.696911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:32.697557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:32.697840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:32.698115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:32.699906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:32.775641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:32.783419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db70b3-750d-45c4-bbca-cd9e6d027242 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:32.789958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1207/2384] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:34.220849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:34.222848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:34.239274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e260b81-ac48-4760-ab48-8b6422b24cd3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:34.244098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:34.244750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:34.245043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:34.245324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:34.247146 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:34.263811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:34.265314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7956f40-b075-46e8-a1c3-6cd5301f6f8a service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:34.277361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1178/2385] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:34.727428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:34.727962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:34.728062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:34.745755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] MFA Rules not processed for user `4df05cbf303c4524a665b429f240a17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:34.747304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:34.858379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1208/2386] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:35.940362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853504c-ad83-483b-9a6d-99c6906bd2a8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:35.940806 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853504c-ad83-483b-9a6d-99c6906bd2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:35.941140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853504c-ad83-483b-9a6d-99c6906bd2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:35.964523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4853504c-ad83-483b-9a6d-99c6906bd2a8 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:35.965532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4853504c-ad83-483b-9a6d-99c6906bd2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:35.978304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1179/2387] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:48:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:35.997444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:36.010373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777cabb-eb6b-4722-a8a1-8bfa5c1d57f8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:36.015390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:36.016153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:36.016452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:36.018970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:36.026014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:36.027652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:36.035771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1209/2388] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:48:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:48:37.433275 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c48f45-ef9e-438f-a60b-d9d14b1837ff None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:37.433768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c48f45-ef9e-438f-a60b-d9d14b1837ff None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:37.433863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c48f45-ef9e-438f-a60b-d9d14b1837ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:37.451456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-21c48f45-ef9e-438f-a60b-d9d14b1837ff None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:37.452640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-21c48f45-ef9e-438f-a60b-d9d14b1837ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:37.460212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1180/2389] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:48:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:37.472370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:37.489124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c313a2e-cf42-4602-90c9-3e6cfcf6dac8 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:37.501903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:37.502573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:37.502859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:37.503165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:37.512325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:37.513905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:37.522561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1210/2390] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:48:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:48:38.320345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1588f6-c690-4432-98e3-4a2318797359 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:38.320880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1588f6-c690-4432-98e3-4a2318797359 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:38.321297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1588f6-c690-4432-98e3-4a2318797359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:38.340697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-3e1588f6-c690-4432-98e3-4a2318797359 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:38.341899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3e1588f6-c690-4432-98e3-4a2318797359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:38.349356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1181/2391] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:48:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:38.362122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:38.374005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dad8e0-f348-4386-a65c-8eec3458456d service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:38.388460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:38.388460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:38.388460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:38.388460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:38.389436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:38.391109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:38.399494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1211/2392] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:48:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:48:39.092357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:39.092850 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:39.093018 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:39.111198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:39.112380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:39.119474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1182/2393] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:48:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:39.132166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:39.143259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399eb2e-fcae-471f-929e-632febe13159 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:39.148791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:39.149426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:39.149714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:39.149995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:39.155023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:39.158757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:39.160374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7006da-5e40-4554-8215-4d2b20bde0fa service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:39.169098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1212/2394] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:48:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:48:42.100705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:42.102260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:42.179277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735bc68-ccd6-4092-a5a1-2bf3f5989536 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:42.185711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:42.186683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:42.186994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:42.187272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] PATH_INFO: `/v3/users/519ddd36b0084a42b18a197b0e5df24f` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:42.188496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=519ddd36b0084a42b18a197b0e5df24f)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:42.190441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:42.207214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:42.496681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519ddd36b0084a42b18a197b0e5df24f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:42.497374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519ddd36b0084a42b18a197b0e5df24f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:42.504119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519ddd36b0084a42b18a197b0e5df24f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:42.648702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519ddd36b0084a42b18a197b0e5df24f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:42.661343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519ddd36b0084a42b18a197b0e5df24f'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:42.667854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1183/2395] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:48:42 2022] DELETE /identity/v3/users/519ddd36b0084a42b18a197b0e5df24f => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:48:42.672354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:42.672844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:42.672957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:42.690496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:42.691728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:42.750756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1213/2396] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:42.760343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:42.761780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:42.822615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9b83a-0fe1-4287-9d0e-d3eb0595cc5e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:42.826227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:42.826877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:42.827200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:42.827483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:42.837477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:42.838911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba97f6-3c0d-461b-b268-92c26c30f924 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:42.848740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1184/2397] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:43.129727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:43.131824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:43.144589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9defad-3b98-47c5-85a1-5a93e22b4aa5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:43.149973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:43.150843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.151157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.151583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] PATH_INFO: `/v3/projects/d2d87fe212654666a76619c9afb14629` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.152772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d87fe212654666a76619c9afb14629)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:43.154755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:43.345640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d87fe212654666a76619c9afb14629', 'request_id': 'req-43c8e833-3ee5-440b-867d-defc6411e98f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:43.351352 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d87fe212654666a76619c9afb14629', 'request_id': 'req-43c8e833-3ee5-440b-867d-defc6411e98f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:43.459758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d87fe212654666a76619c9afb14629', 'request_id': 'req-43c8e833-3ee5-440b-867d-defc6411e98f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:43.471261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] The token cache is being invalidate because project d2d87fe212654666a76619c9afb14629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:48:43.471548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:43.472166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-43c8e833-3ee5-440b-867d-defc6411e98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:48:43.473883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1214/2398] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:48:43 2022] DELETE /identity/v3/projects/d2d87fe212654666a76619c9afb14629 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:48:43.479203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.479203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.479681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.502211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:43.503051 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:43.575078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1185/2399] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:43.577074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.577363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.577644 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.596666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:43.597880 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:43.606457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1215/2400] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:43.610752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:43.621119 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f5a2-49e7-483a-a33f-8f2cdb2c391b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:43.628028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:43.628694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.628981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.629262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.630046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:48:43.630355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:43.631978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:43.643847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1186/2401] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:48:43.648118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:43.657359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70073c01-d860-4d4c-bc88-28b1e289a819 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:43.662024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:43.663230 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.663515 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.663792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.664486 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:43.666387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:43.798688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1216/2402] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:48:43 2022] POST /identity/v3/projects => generated 391 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:48:43.802763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:43.813244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edf872-dbed-44d7-988e-397c1609016f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:43.832974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:43.868833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:43.869518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.869804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.870079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.870796 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:43.872749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:43.962910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1187/2403] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:48:43 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:48:43.966841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:43.977545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19aff6-cf1d-43b8-b7b5-20c02882af48 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:43.982108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:43.982757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:43.983077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:43.983354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:43.984185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:43.985766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:43.996288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1217/2404] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:48:44.000489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:44.010609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb795d3-681c-4666-b838-d3b4ea1c7ccc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:44.015311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:44.025152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:44.025152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:44.025152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:44.025152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:44.025152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6f86-3e3c-4ddc-9e04-84950b478a62 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:44.030629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1188/2405] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:48:44.035109 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:44.046115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846be75-d5c1-4f13-be14-d8e2855bd67e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:44.050835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:44.051761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:44.052042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:44.052320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] PATH_INFO: `/v3/projects/e9382553c2b04cd2bcd6303d47eb4db4/users/69f07b8e4a784fb4985169f16e2dac7a/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:44.067189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9382553c2b04cd2bcd6303d47eb4db4, user_id=69f07b8e4a784fb4985169f16e2dac7a, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:44.070092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:44.190886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1218/2406] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:48:44 2022] PUT /identity/v3/projects/e9382553c2b04cd2bcd6303d47eb4db4/users/69f07b8e4a784fb4985169f16e2dac7a/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:48:44.201618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:44.201908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:44.202191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:44.228601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] MFA Rules not processed for user `69f07b8e4a784fb4985169f16e2dac7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:48:44.229857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:44.292920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1189/2407] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:48:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:48:44.304099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:44.305813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:44.321018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:44.364813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938aee-49c9-4c7e-89b7-9b05cbd248f9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:44.371285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:44.371285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:44.371285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:44.371285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:44.377909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:44.379416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:44.388611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1219/2408] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:47.052509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:48:47.054093 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:47.128947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9984b8-99af-496a-8d8c-d6e7c04a4c64 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:48:47.134321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:48:47.141637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:48:47.141637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:48:47.141637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:48:47.141637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:48:47.163399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:47.197194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:48:47.198808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:48:47.208152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1190/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:48:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:48:54.431374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=20 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:48:57.510530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=16 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:03.710316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:03.711820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:03.729394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d94e-91ed-4900-8964-bd69b5957c49 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:03.735176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:03.735828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:03.736110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:03.736386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:03.738206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:03.751298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:03.752671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:03.764132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1220/2410] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:03.835291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:03.836709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:03.906688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41a80-0162-4b65-a622-2c273a096115 service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:03.910618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:03.911292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:03.911573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:03.911847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:03.913706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:03.928138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:03.929577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97d0da-cfbc-473a-be76-6d5df0aa5bbd service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:03.937517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1191/2411] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:05.754737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:05.820842 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46055-98f9-4f92-b55b-3a2124237aac service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:05.825448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:05.826321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:05.826610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:05.826889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] PATH_INFO: `/v3/users/69f07b8e4a784fb4985169f16e2dac7a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:05.828133 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f07b8e4a784fb4985169f16e2dac7a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:05.830055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:06.403917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f07b8e4a784fb4985169f16e2dac7a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:06.404563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f07b8e4a784fb4985169f16e2dac7a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:06.411817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f07b8e4a784fb4985169f16e2dac7a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:06.556160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f07b8e4a784fb4985169f16e2dac7a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:06.569736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f07b8e4a784fb4985169f16e2dac7a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:06.575826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1221/2412] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:49:05 2022] DELETE /identity/v3/users/69f07b8e4a784fb4985169f16e2dac7a => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:06.580408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:06.580872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:06.580969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:06.598242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:06.599489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:06.658523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1192/2413] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:06.668125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:06.669605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:06.726734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec333-974a-4ec6-b761-64b1a62ebdf7 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:06.730333 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:06.731005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:06.731292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:06.731572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:06.740874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:06.742293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:06.751864 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1222/2414] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:07.289013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:07.290443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:07.302905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2554a2-1fa3-4677-878b-2c5a0d0a16b1 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:07.307915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:07.311370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:07.311370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:07.311370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] PATH_INFO: `/v3/projects/e9382553c2b04cd2bcd6303d47eb4db4` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:07.311370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9382553c2b04cd2bcd6303d47eb4db4)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:07.312489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:07.459331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9382553c2b04cd2bcd6303d47eb4db4', 'request_id': 'req-a24d3f85-6091-4f13-afa9-84a13e5a7369'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:07.464612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9382553c2b04cd2bcd6303d47eb4db4', 'request_id': 'req-a24d3f85-6091-4f13-afa9-84a13e5a7369'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:07.572934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9382553c2b04cd2bcd6303d47eb4db4', 'request_id': 'req-a24d3f85-6091-4f13-afa9-84a13e5a7369'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:07.584337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] The token cache is being invalidate because project e9382553c2b04cd2bcd6303d47eb4db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:49:07.585063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:07.585331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:07.586894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1193/2415] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:49:07 2022] DELETE /identity/v3/projects/e9382553c2b04cd2bcd6303d47eb4db4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:10.951086 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:10.952439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:10.963944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:11.024734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f925c67-af00-4c4f-b73d-e4bbe558f605 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:11.087944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:11.088081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:11.088284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:11.088567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:11.090629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:11.148147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:11.149774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:11.159916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1223/2416] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:17.172601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:17.174085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:17.191355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d3f85-6091-4f13-afa9-84a13e5a7369 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:17.197727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:17.198610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:17.198753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:17.199070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:17.200857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:17.213229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:17.214812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcbd7b-588f-4378-bb56-fef40ae687a2 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:17.225714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1194/2417] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:21.955939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:21.957417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:22.008498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:22.019528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55889bda-1a87-4bbc-b8d6-65ebeb852e98 service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:22.027675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:22.027675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:22.027675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:22.027675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:22.028084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:22.091088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:22.092638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:22.102641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1224/2418] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:25.465484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:25.476586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:25.476586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:25.491296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] MFA Rules not processed for user `8b5a188cee0d4727a14153088b8d95c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:25.492523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:25.499842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1195/2419] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:25.807522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:25.808965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:25.868802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f28ba-0707-494a-a9f0-889c801f271c service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:25.872458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:25.873105 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:25.873392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:25.873669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:25.875606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:25.884918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:25.886568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:25.899659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1225/2420] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:26.334065 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:26.335590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:26.345257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:26.408398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ee10e-5fe6-4420-81f1-3f5ff9f0c961 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:26.413928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:26.414573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:26.414863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:26.415179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:26.416984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:26.428682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:26.430247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:26.439980 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1196/2421] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:32.529278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:32.531549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:32.619166 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca281a7a-6bcc-4c85-b757-ed0ad5cfefc8 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:32.624294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:32.625373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:32.625772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:32.626063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] PATH_INFO: `/v3/users/a7fd310fe0254401b10f6b1517994233` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:32.627342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fd310fe0254401b10f6b1517994233)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:32.630193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:32.928290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:32.931284 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fd310fe0254401b10f6b1517994233'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:32.931989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fd310fe0254401b10f6b1517994233'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:32.939095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fd310fe0254401b10f6b1517994233'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:33.046750 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fd310fe0254401b10f6b1517994233'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:33.064528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1fdee92-c469-4531-b162-d7d50fa10b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fd310fe0254401b10f6b1517994233'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:33.074402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1226/2422] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:49:32 2022] DELETE /identity/v3/users/a7fd310fe0254401b10f6b1517994233 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:49:33.706400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:33.719120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8988a-03ae-4a42-aed2-a51569da4b80 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:33.779715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:33.780572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:33.780858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:33.781132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] PATH_INFO: `/v3/projects/da06c03197114259aac4bf5444485d51` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:33.782307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da06c03197114259aac4bf5444485d51)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:33.784232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.040705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da06c03197114259aac4bf5444485d51', 'request_id': 'req-d495c200-3bc2-4761-a8a3-12ccff8da2a2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:34.046134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da06c03197114259aac4bf5444485d51', 'request_id': 'req-d495c200-3bc2-4761-a8a3-12ccff8da2a2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:34.257557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da06c03197114259aac4bf5444485d51', 'request_id': 'req-d495c200-3bc2-4761-a8a3-12ccff8da2a2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:34.269239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] The token cache is being invalidate because project da06c03197114259aac4bf5444485d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:49:34.269532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:34.270162 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d495c200-3bc2-4761-a8a3-12ccff8da2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:34.271731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1197/2423] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:49:33 2022] DELETE /identity/v3/projects/da06c03197114259aac4bf5444485d51 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:34.276721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.277213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.277332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.295188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:34.296329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:34.355271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1227/2424] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:34.363818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.364113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.364397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.382990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:34.384199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:34.392185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1198/2425] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:34.396459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.405973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c13499-a81c-47a8-aa09-be1eef492db3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.410639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.411326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.411613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.411889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.412614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:49:34.412924 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.414532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.426505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1228/2426] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:49:34.430758 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.439952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5211ce-57e8-4f4b-8c60-c85006cc432d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.445489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.446132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.446417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.446697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.447453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.449305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.589561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:34.596785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1199/2427] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:49:34 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:34.601345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.611638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def525f7-5c18-4194-8e24-6b651f374fbc admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.677874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.678559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.678853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.679164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.679869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.681838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.852926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1229/2428] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:49:34 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:49:34.857010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.867723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88ccc-d50e-4948-8827-d69835ab3229 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.872222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.872885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.873429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.873537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.874260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.875911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.886858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1200/2429] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:34.893456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.903423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247e929-6093-4d9f-b9fa-89832c2283d2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.907959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.908609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.908900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.909176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.910402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.912014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84511c5a-e71a-40f8-8db0-a0bd8ca6cef1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:34.922559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1230/2430] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:34.928257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:34.938038 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60317d4-8524-46ef-b725-135d7c927a56 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:34.942792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:34.943694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:34.943984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:34.944264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] PATH_INFO: `/v3/projects/6329f898314d44d2b6c264d8224a8c46/users/ab96fb75e716433f857cb9a4c0d7fb2a/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:34.959407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6329f898314d44d2b6c264d8224a8c46, user_id=ab96fb75e716433f857cb9a4c0d7fb2a, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:34.962883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:35.114235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1201/2431] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:34 2022] PUT /identity/v3/projects/6329f898314d44d2b6c264d8224a8c46/users/ab96fb75e716433f857cb9a4c0d7fb2a/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:49:35.117914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:35.118208 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:35.118488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:35.136254 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:35.137632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:35.157801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:35.197398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1231/2432] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:49:35.207612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:35.209431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:35.267361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d18e2-9c75-4bb6-a550-b6ddeb5a75f2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:35.271717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:35.272357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:35.272642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:35.272919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:35.282797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:35.284279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4e055-64c4-4107-a681-030803795d17 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:35.294568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1202/2433] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:39.683285 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:39.685453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:39.917848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49619f0-50d6-4f5c-bad2-60bafb7cda8e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:39.926077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:39.928124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:39.928492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:39.928932 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:39.931184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:39.951859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:39.957116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:39.993951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1232/2434] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:42.084755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.085253 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.085346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.112465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] MFA Rules not processed for user `ab96fb75e716433f857cb9a4c0d7fb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:42.114328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:42.192552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1203/2435] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:42.193087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.203762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a875-66e6-4fb3-9f2e-5229f32ee3e4 service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.208507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.209167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.209449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.209725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.210430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.212359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.350682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1233/2436] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:49:42 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:42.361216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.372353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bee36-ff97-47d0-9a6e-716775d05ba8 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.427531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:42.431221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.431905 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.432193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.432474 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.433319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.435317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.566133 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1204/2437] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:49:42 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:42.570338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.581682 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76946677-68e8-4c93-bb53-b8e15002e608 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.586378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.591952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.591952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.591952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.591952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.591952 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.602363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1234/2438] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:42.606564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.617239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ddfd-5071-4fca-8ee1-8c736fc42f15 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.622537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.623577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.623756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.624001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] PATH_INFO: `/v3/projects/e606c1202c254c32ba913a2a49cf30ca/users/b58ebc02f5224524a617f13b2faed414/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.645125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e606c1202c254c32ba913a2a49cf30ca, user_id=b58ebc02f5224524a617f13b2faed414, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.648021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.766017 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1205/2439] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:42 2022] PUT /identity/v3/projects/e606c1202c254c32ba913a2a49cf30ca/users/b58ebc02f5224524a617f13b2faed414/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:42.766222 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.776282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6f42b-fcad-4f39-b107-0d981c55b8a8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.837198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.839687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.839687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.839687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.839687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.840882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.852831 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1235/2440] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:49:42.861924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:42.873215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e84cb-fc60-4d92-89f3-1c9f40f05862 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:42.877989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:42.879562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:42.879848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:42.880125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] PATH_INFO: `/v3/projects/e606c1202c254c32ba913a2a49cf30ca/users/b58ebc02f5224524a617f13b2faed414/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:42.882148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e606c1202c254c32ba913a2a49cf30ca, user_id=b58ebc02f5224524a617f13b2faed414, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:42.884884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7512de-d753-4e8f-aff7-0c841a585778 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:42.898989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1206/2441] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:42 2022] PUT /identity/v3/projects/e606c1202c254c32ba913a2a49cf30ca/users/b58ebc02f5224524a617f13b2faed414/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:47.767667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:47.769348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:47.804235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:47.849625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372fdac-dfcf-487f-a04d-a5491a9da37d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:47.915328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:47.915948 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:47.916277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:47.916566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:47.918583 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:47.990324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:47.991913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:48.003605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1236/2442] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:50.753393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8007b-5824-43fc-a3fb-3faa246fbe07 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:50.753871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8007b-5824-43fc-a3fb-3faa246fbe07 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:50.753871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8007b-5824-43fc-a3fb-3faa246fbe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:50.806617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-09e8007b-5824-43fc-a3fb-3faa246fbe07 None None] MFA Rules not processed for user `b58ebc02f5224524a617f13b2faed414`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:50.808379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-09e8007b-5824-43fc-a3fb-3faa246fbe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:50.911025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1207/2443] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:50.937068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:50.939729 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:51.048960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96463-aa15-41ba-8e0a-e50a0abf700e service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:51.058298 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:51.059268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:51.059710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:51.060224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:51.224913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:51.226908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdf230-870c-48c7-af2f-b836a42c3982 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:51.248803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1237/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:51.359457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:53.955076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:53.956372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:53.956836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:53.959261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1208/2445] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:49:53.973069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:53.973668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:53.977437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:54.040986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] MFA Rules not processed for user `02a46335d4b248048b82d04e3665fe55`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:54.043985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:54.146588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1238/2446] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:54.158505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:54.170679 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:54.235164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:54.235164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:54.238043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d3d4-b6d3-4dae-bfba-16e7a282874c None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:54.244919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:54.245826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:54.246581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:54.247120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:54.260005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59286c-ae18-4138-813f-d14d189f4ab9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:54.260147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:54.263368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:54.270768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:54.271721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:54.272012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:54.272293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] PATH_INFO: `/v3/users/2d0d5723417d4b60bc51e47f28d09902` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:54.274223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0d5723417d4b60bc51e47f28d09902)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:54.279997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:54.297931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1209/2447] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:54.530358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0d5723417d4b60bc51e47f28d09902'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:54.531240 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0d5723417d4b60bc51e47f28d09902'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:54.539237 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0d5723417d4b60bc51e47f28d09902'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:54.727367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0d5723417d4b60bc51e47f28d09902'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:54.746263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-32e37780-7535-411b-bea0-0bb799ed1b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0d5723417d4b60bc51e47f28d09902'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:54.756472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1239/2448] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:49:54 2022] DELETE /identity/v3/users/2d0d5723417d4b60bc51e47f28d09902 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:49:55.047278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:55.063450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556889f7-e9da-4503-81c1-ea71fbc06ce8 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:55.154000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:55.155068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.155511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.155853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] PATH_INFO: `/v3/projects/7153c09b2d0b4805ad0360afb7ecd3ca` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.157391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7153c09b2d0b4805ad0360afb7ecd3ca)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:55.159984 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:55.189466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:55.298705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7153c09b2d0b4805ad0360afb7ecd3ca', 'request_id': 'req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:55.305327 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7153c09b2d0b4805ad0360afb7ecd3ca', 'request_id': 'req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:55.388397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7153c09b2d0b4805ad0360afb7ecd3ca', 'request_id': 'req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:55.411407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] The token cache is being invalidate because project 7153c09b2d0b4805ad0360afb7ecd3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:49:55.411886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:55.412899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-8688d0e4-3b33-4cbb-9f25-ad26770aa8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:49:55.415965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1210/2449] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:49:55 2022] DELETE /identity/v3/projects/7153c09b2d0b4805ad0360afb7ecd3ca => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:55.422561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.423196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.423196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.452974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:55.454550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:55.470151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:55.536082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1240/2450] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:55.546379 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.546762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.547649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.572035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:55.573271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:55.581884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1211/2451] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:55.586236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:55.597845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4a85b-8ffd-4bd4-ba0b-8122bee1887a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:55.605588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:55.605588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.605588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.605588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.606088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:49:55.606088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:55.606780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:55.619250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1241/2452] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:49:55.623709 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:55.634881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55dc8-8a00-455c-b38c-978bd89dcd8a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:55.640208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:55.640922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.641309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.641741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.642467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:55.645048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:55.808304 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1212/2453] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:49:55 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:55.812837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:55.827660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddfc88-1b07-437a-a1d3-27fdbe0cfd34 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:55.901206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:55.901886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:55.902180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:55.902661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:55.903358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:55.905590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:56.049185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1242/2454] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:49:55 2022] POST /identity/v3/users => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:56.077439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:56.089059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb5225-7ca4-41d6-80b1-c326c3768555 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:56.097420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:56.098132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.098483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.098771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.099754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:56.101585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:56.113035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1213/2455] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:56.117143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:56.128065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae87b3-1c71-412c-8fb6-ab472e37aa41 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:56.133088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:56.133840 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.136784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.136784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.136784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:56.137074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d02d1-cb85-4f4f-be39-004941d09d3b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:56.148872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1243/2456] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:56.153247 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:56.164477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f341d49-c7eb-4161-8c13-1afbc20d7e7e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:56.169836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:56.172225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.172531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.172811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] PATH_INFO: `/v3/projects/a3a406149d0b46b1811e9b8e33489258/users/28372172e3c44cc8b6e01ffaa2bae5cb/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.195782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a406149d0b46b1811e9b8e33489258, user_id=28372172e3c44cc8b6e01ffaa2bae5cb, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:56.195782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:56.197603 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0d85-b6c9-419c-8dcf-fd4bec184769 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.198057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0d85-b6c9-419c-8dcf-fd4bec184769 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.198415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0d85-b6c9-419c-8dcf-fd4bec184769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.217923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-87ee0d85-b6c9-419c-8dcf-fd4bec184769 None None] MFA Rules not processed for user `ab96fb75e716433f857cb9a4c0d7fb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:56.220315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-87ee0d85-b6c9-419c-8dcf-fd4bec184769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:56.278907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1244/2457] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:56.310048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1214/2458] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:56 2022] PUT /identity/v3/projects/a3a406149d0b46b1811e9b8e33489258/users/28372172e3c44cc8b6e01ffaa2bae5cb/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:56.313446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceee1d-9020-4a43-9be7-d326779258e0 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.313769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceee1d-9020-4a43-9be7-d326779258e0 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.314039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eceee1d-9020-4a43-9be7-d326779258e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.348101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4eceee1d-9020-4a43-9be7-d326779258e0 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:56.349340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4eceee1d-9020-4a43-9be7-d326779258e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:56.423386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1245/2459] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:56.430819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:56.432684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:56.514081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb387-e087-4c5c-a3fe-ac91d748ebf0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:56.517967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:56.519185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:56.519473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:56.519773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:56.530472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:56.532865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:56.548428 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1215/2460] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:49:58.680600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa9c3-b917-498b-8c51-912900334836 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:58.681211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa9c3-b917-498b-8c51-912900334836 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:58.681569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa9c3-b917-498b-8c51-912900334836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:58.709517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-382aa9c3-b917-498b-8c51-912900334836 None None] MFA Rules not processed for user `28372172e3c44cc8b6e01ffaa2bae5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:49:58.710843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-382aa9c3-b917-498b-8c51-912900334836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:49:58.789853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1246/2461] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:49:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:49:58.794464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:58.804447 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a028-3122-414b-b958-577265e602b0 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:58.809089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:58.809739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:58.810022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:58.810297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:58.811014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:58.812899 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:58.984853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1216/2462] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:49:58 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:58.989678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa9c3-b917-498b-8c51-912900334836 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:59.000507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa9c3-b917-498b-8c51-912900334836 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:59.056455 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:59.057132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:59.057419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:59.057697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:59.058387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:59.060412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:59.287955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1247/2463] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:49:58 2022] POST /identity/v3/users => generated 309 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:49:59.291057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:59.303947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7510-1164-4875-a894-d18c4ab5ec5e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:59.308634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:59.309297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:59.309580 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:59.309858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:59.310619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:59.312255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:59.323922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1217/2464] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:59.328654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:59.338863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210d7f-47c6-4d33-bf69-0b74e728379b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:59.343782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:59.344658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:59.344947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:59.345215 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] PATH_INFO: `/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7/users/d95cdc5c43de4b8398009f057ac67d0a/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:59.362909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbdd79b0c224d79b9dfabeb894802b7, user_id=d95cdc5c43de4b8398009f057ac67d0a, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:59.365597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:59.480424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:59.489838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1248/2465] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:59 2022] PUT /identity/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7/users/d95cdc5c43de4b8398009f057ac67d0a/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:49:59.494001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:59.504499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c8290-bb09-4a32-9ffc-c166ee138c60 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:59.509357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:49:59.564838 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:59.565429 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:59.565718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:59.566000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:59.566989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:59.568627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51188bc3-45fe-45a8-a69f-1b6a439ced47 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:59.578935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1218/2466] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:49:59.583097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:49:59.593754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec558a8d-d963-4864-ba9b-39f40104b798 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:49:59.599198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:49:59.600084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:49:59.600368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:49:59.600643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] PATH_INFO: `/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7/users/d95cdc5c43de4b8398009f057ac67d0a/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:49:59.602604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbdd79b0c224d79b9dfabeb894802b7, user_id=d95cdc5c43de4b8398009f057ac67d0a, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:49:59.605283 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:49:59.616395 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1249/2467] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:49:59 2022] PUT /identity/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7/users/d95cdc5c43de4b8398009f057ac67d0a/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:01.550417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:01.553622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:01.553622 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:01.586627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] MFA Rules not processed for user `d95cdc5c43de4b8398009f057ac67d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:01.593617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:01.665754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1219/2468] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:03.220442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:03.244908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:03.304465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44c28d-9ca0-4163-9c5d-6c96f8f78fd5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:03.309934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:03.310859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:03.311177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:03.311458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] PATH_INFO: `/v3/users/28372172e3c44cc8b6e01ffaa2bae5cb` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:03.312789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28372172e3c44cc8b6e01ffaa2bae5cb)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:03.314728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:03.620809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28372172e3c44cc8b6e01ffaa2bae5cb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:03.621500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28372172e3c44cc8b6e01ffaa2bae5cb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:03.628579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28372172e3c44cc8b6e01ffaa2bae5cb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:03.770756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28372172e3c44cc8b6e01ffaa2bae5cb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:03.783412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28372172e3c44cc8b6e01ffaa2bae5cb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:03.789879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1250/2469] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:03 2022] DELETE /identity/v3/users/28372172e3c44cc8b6e01ffaa2bae5cb => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:04.075693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:04.087060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ecedb-62ed-40d9-b617-249821588991 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:04.148305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:04.149190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:04.149472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:04.149857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] PATH_INFO: `/v3/projects/a3a406149d0b46b1811e9b8e33489258` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:04.150992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a406149d0b46b1811e9b8e33489258)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:04.154337 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:04.302351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:04.315441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a406149d0b46b1811e9b8e33489258', 'request_id': 'req-82d5c835-2a75-4550-bb64-8da048501776'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.321293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a406149d0b46b1811e9b8e33489258', 'request_id': 'req-82d5c835-2a75-4550-bb64-8da048501776'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.418024 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a406149d0b46b1811e9b8e33489258', 'request_id': 'req-82d5c835-2a75-4550-bb64-8da048501776'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.430422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] The token cache is being invalidate because project a3a406149d0b46b1811e9b8e33489258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:04.430714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.431365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.432950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1220/2470] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:04 2022] DELETE /identity/v3/projects/a3a406149d0b46b1811e9b8e33489258 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:04.436942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:04.438390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:04.453423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960366f1-cf2f-438b-8552-f6dc8daf68d6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:04.513430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:04.514319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:04.514606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:04.514900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] PATH_INFO: `/v3/users/d95cdc5c43de4b8398009f057ac67d0a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:04.516232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95cdc5c43de4b8398009f057ac67d0a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:04.518303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:04.748419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:04.819834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95cdc5c43de4b8398009f057ac67d0a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.820935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95cdc5c43de4b8398009f057ac67d0a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.827862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95cdc5c43de4b8398009f057ac67d0a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.913537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95cdc5c43de4b8398009f057ac67d0a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.928124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-160867e7-4880-483d-8900-43acb6f1312c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95cdc5c43de4b8398009f057ac67d0a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:04.934518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1251/2471] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:04 2022] DELETE /identity/v3/users/d95cdc5c43de4b8398009f057ac67d0a => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:05.343612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:05.345089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:05.362604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c835-2a75-4550-bb64-8da048501776 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:05.421279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:05.422074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:05.422361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:05.422639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] PATH_INFO: `/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:05.423865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbdd79b0c224d79b9dfabeb894802b7)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:05.425818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:05.630535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbdd79b0c224d79b9dfabeb894802b7', 'request_id': 'req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:05.636511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbdd79b0c224d79b9dfabeb894802b7', 'request_id': 'req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:05.788361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbdd79b0c224d79b9dfabeb894802b7', 'request_id': 'req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:05.813690 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] The token cache is being invalidate because project dcbdd79b0c224d79b9dfabeb894802b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:05.814011 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:05.815230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d1c6a16c-61d0-4e54-a8c2-17c2d94eddc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:05.817422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1221/2472] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:05 2022] DELETE /identity/v3/projects/dcbdd79b0c224d79b9dfabeb894802b7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:05.822274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:05.823068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:05.823543 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:05.842968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:05.844363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:05.917780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1252/2473] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:05.926243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:05.926792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:05.928028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:05.946769 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:05.948000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:05.955759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1222/2474] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:05.960176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:05.969939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1c4ef-e7e7-4ced-b78d-8207d5602a1a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:05.974542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:05.975228 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:05.975522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:05.975804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:05.976531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:50:05.976844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:05.978601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:05.996879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1253/2475] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:50:05.997921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.008539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602d18f-4f73-4c4c-ac19-544ce41a40af None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.013415 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.014071 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.014356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.014794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.015649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.017551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.149990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1223/2476] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:05 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:06.154184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.164906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a21a49-b7ea-471c-88c4-cff8004488c1 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.224174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.224767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.225056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.225337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.226038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.228003 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.371062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1254/2477] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:06 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:06.375135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.385441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724d16-cb06-4c4b-b165-86e5d8109e70 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.390046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.390703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.391023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.391311 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.392099 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.393705 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.404625 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1224/2478] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:06.409006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.419659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11c20-efdc-4e4a-8108-2d1857563357 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.425174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.425838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.426168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.426464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.427248 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.428860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebb852-5c8c-40c9-a9bc-95387fd57686 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.439731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1255/2479] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:06.444193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.454044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4440c-4274-4c11-a708-99cdeead0a7a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.459408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.463674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.463674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.463674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] PATH_INFO: `/v3/projects/9652ce52870248bea2ec9e2d4a33cfda/users/ff6c62d769af458196769415e9cc2165/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.464256 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:06.477936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9652ce52870248bea2ec9e2d4a33cfda, user_id=ff6c62d769af458196769415e9cc2165, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.480530 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.611616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1225/2480] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:06 2022] PUT /identity/v3/projects/9652ce52870248bea2ec9e2d4a33cfda/users/ff6c62d769af458196769415e9cc2165/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:06.621887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.622183 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.622468 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.649571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] MFA Rules not processed for user `ff6c62d769af458196769415e9cc2165`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:06.650784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:06.716353 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1256/2481] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:06.720803 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.784737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff4365-a53c-44ab-9bb9-ba4aef0373f1 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:06.789585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:06.790248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:06.790532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:06.790805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:06.791537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:06.793575 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:06.879942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:06.964600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1226/2482] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:06 2022] POST /identity/v3/projects => generated 363 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:06.965836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:06.980316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9c9e4-8587-4676-ba5e-4ff910339637 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.056127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.056877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.057096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.057379 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.058079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.060345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.269887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1257/2483] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:06 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:07.275844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:07.286712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c894ba-b9ba-4676-a869-5e4834a1c187 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.291339 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.292006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.292301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.292593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.293344 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.295202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.307405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1227/2484] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:07.311396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:07.327312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac728c-4bdf-4474-a7fc-b9ec641f7728 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.336916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.337837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.338129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.338412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] PATH_INFO: `/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d/users/c6602fd415d2470d8e748d7334491715/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.354685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2e30fd03424b5d8d824a9cf35ee30d, user_id=c6602fd415d2470d8e748d7334491715, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.357557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.491636 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1258/2485] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:07 2022] PUT /identity/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d/users/c6602fd415d2470d8e748d7334491715/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:07.495693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:07.508486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4592d95-cfd7-4685-979e-144643dd3385 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.565225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.571255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.571255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.571255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.571255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.571255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1660e-bf4b-46d0-89b7-c77f0675cb7e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.580171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1228/2486] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:07.584158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:07.594755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b7042-40dd-4ec3-8ab3-00d1e85e75b8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.603176 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.604093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.605607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.605607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] PATH_INFO: `/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d/users/c6602fd415d2470d8e748d7334491715/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.609292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2e30fd03424b5d8d824a9cf35ee30d, user_id=c6602fd415d2470d8e748d7334491715, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.613456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.650726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1259/2487] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:07 2022] PUT /identity/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d/users/c6602fd415d2470d8e748d7334491715/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:07.652134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.652431 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.652715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.693361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] MFA Rules not processed for user `c6602fd415d2470d8e748d7334491715`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:07.694851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:07.784055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1229/2488] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:07.795792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:07.797527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:07.859582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7bc6-ae78-4fb0-bac1-516809892465 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:07.863922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:07.864593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:07.864882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:07.865162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:07.875148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:07.876606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a62ea0-06d2-4f4e-be16-04ec9da50a9a service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:07.887393 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1260/2489] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:07 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:08.445490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:08.788782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:09.059456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:09.128105 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa86a0-4d24-4ea1-835e-f985e7aefa39 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:09.132892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:09.133978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:09.134274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:09.134555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] PATH_INFO: `/v3/users/ff6c62d769af458196769415e9cc2165` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:09.135869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6c62d769af458196769415e9cc2165)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:09.137883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:09.395944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6c62d769af458196769415e9cc2165'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:09.396634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6c62d769af458196769415e9cc2165'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:09.407004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6c62d769af458196769415e9cc2165'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:09.478902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6c62d769af458196769415e9cc2165'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:09.492384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6c62d769af458196769415e9cc2165'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:09.498535 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1230/2490] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:09 2022] DELETE /identity/v3/users/ff6c62d769af458196769415e9cc2165 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:09.504007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4afc0-9a95-4f0d-927e-e366d50c4d9b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:09.504485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4afc0-9a95-4f0d-927e-e366d50c4d9b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:09.504667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4afc0-9a95-4f0d-927e-e366d50c4d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:09.533907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1ee4afc0-9a95-4f0d-927e-e366d50c4d9b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:09.535188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1ee4afc0-9a95-4f0d-927e-e366d50c4d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:09.597116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1261/2491] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:09.608124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:09.609554 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:09.630192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:09.630493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:09.630777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:09.648491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:09.649744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:09.669794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4e23-03d5-46f6-953f-6ef068b6d5d3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:09.673455 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:09.674761 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:09.675096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:09.675489 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:09.684687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:09.686267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:09.697996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1231/2492] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:09.704549 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1262/2493] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:50:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:10.014282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:10.015721 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:10.028081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:10.089815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce5b5-eda5-4860-a396-4d3dfefbaf41 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:10.095054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:10.096220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:10.096510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:10.096792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:10.105585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:10.107213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:10.116183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1232/2494] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:50:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:50:10.338209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:10.339666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:10.354485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1731983-7791-4608-8a7a-fb9c88ce0444 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:10.359143 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:10.360012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:10.360299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:10.360577 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] PATH_INFO: `/v3/projects/9652ce52870248bea2ec9e2d4a33cfda` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:10.361777 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9652ce52870248bea2ec9e2d4a33cfda)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:10.363702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:10.531134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9652ce52870248bea2ec9e2d4a33cfda', 'request_id': 'req-307dbf97-17ea-4150-8b2f-f8c94b9874dc'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.538367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9652ce52870248bea2ec9e2d4a33cfda', 'request_id': 'req-307dbf97-17ea-4150-8b2f-f8c94b9874dc'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.639338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9652ce52870248bea2ec9e2d4a33cfda', 'request_id': 'req-307dbf97-17ea-4150-8b2f-f8c94b9874dc'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.651392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] The token cache is being invalidate because project 9652ce52870248bea2ec9e2d4a33cfda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:10.651685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.652321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.653869 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1263/2495] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:10 2022] DELETE /identity/v3/projects/9652ce52870248bea2ec9e2d4a33cfda => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:10.657847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:10.668406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c633-9e94-4647-9c6d-fed451ea6a96 service swift] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:10.724793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:10.725686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:10.725968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:10.726243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] PATH_INFO: `/v3/users/c6602fd415d2470d8e748d7334491715` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:10.727480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6602fd415d2470d8e748d7334491715)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:10.729421 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:10.993284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6602fd415d2470d8e748d7334491715'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:10.994212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6602fd415d2470d8e748d7334491715'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:11.003197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6602fd415d2470d8e748d7334491715'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:11.153375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6602fd415d2470d8e748d7334491715'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:11.167378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7703618a-cc80-47e6-8e92-b50412dafe35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6602fd415d2470d8e748d7334491715'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:11.176662 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1233/2496] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:10 2022] DELETE /identity/v3/users/c6602fd415d2470d8e748d7334491715 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:11.852042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:11.853516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:11.868793 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dbf97-17ea-4150-8b2f-f8c94b9874dc admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:11.890498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:11.926442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:11.927360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:11.927651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:11.927931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] PATH_INFO: `/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:11.929165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2e30fd03424b5d8d824a9cf35ee30d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:11.931115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:12.138403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2e30fd03424b5d8d824a9cf35ee30d', 'request_id': 'req-5dbf44d5-19e6-4014-87e5-9bee02a3814d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:12.144525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2e30fd03424b5d8d824a9cf35ee30d', 'request_id': 'req-5dbf44d5-19e6-4014-87e5-9bee02a3814d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:12.410574 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2e30fd03424b5d8d824a9cf35ee30d', 'request_id': 'req-5dbf44d5-19e6-4014-87e5-9bee02a3814d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:12.423001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] The token cache is being invalidate because project 6d2e30fd03424b5d8d824a9cf35ee30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:12.423523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:12.424915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-5dbf44d5-19e6-4014-87e5-9bee02a3814d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:12.427355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1264/2497] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:11 2022] DELETE /identity/v3/projects/6d2e30fd03424b5d8d824a9cf35ee30d => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:12.437448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:12.437997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:12.437997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:12.458533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:12.459788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:12.521163 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1234/2498] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:12.529081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:12.529557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:12.529930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:12.548794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:12.550012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:12.558387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1265/2499] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:12.562978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:12.573083 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cb6de-773d-46ad-8aa4-3c73a4ded386 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:12.578237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:12.579191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:12.579338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:12.579521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:12.580828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:50:12.581142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:12.582771 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:12.594629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1235/2500] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:50:12.599933 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:12.609099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d59c1-975e-4c5d-ba63-0926cf67f2c8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:12.613966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:12.614634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:12.615024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:12.615844 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:12.616192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:12.618123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:12.818800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1266/2501] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:12 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:12.822716 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:12.833386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a027d1-1f7f-4839-9c83-3fc915032863 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:12.890217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:12.891002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:12.891293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:12.891603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:12.892460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:12.894801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:13.138669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1236/2502] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:12 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:13.142096 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:13.155664 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cdeab-677d-48e1-834e-2f372921aab8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:13.161130 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:13.162032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:13.162479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:13.162771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:13.163564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:13.165219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:13.176441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1267/2503] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:13.180908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:13.192931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0e11-2519-4502-850a-8408ea9b57db admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:13.197603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:13.198262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:13.198547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:13.198821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:13.199590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:13.201188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07393500-d0bb-4a03-9911-3b34fa5394dc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:13.212481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1237/2504] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:13.216689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:13.227177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa924b-05ce-4e16-8ed7-65854e97b730 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:13.232518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:13.233462 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:13.233706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:13.233989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] PATH_INFO: `/v3/projects/79f4863513bb439ea18976c9953d065e/users/901f81c893aa40de9598bc9676e52028/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:13.249467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f4863513bb439ea18976c9953d065e, user_id=901f81c893aa40de9598bc9676e52028, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:13.252876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:13.279492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:13.442015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1268/2505] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:13 2022] PUT /identity/v3/projects/79f4863513bb439ea18976c9953d065e/users/901f81c893aa40de9598bc9676e52028/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:13.457464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:13.458126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:13.458126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:13.516207 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] MFA Rules not processed for user `901f81c893aa40de9598bc9676e52028`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:13.518770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:13.622131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1238/2506] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:13.626947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:13.711002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2614d-1039-471b-9c2f-41404b337aba admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:13.716151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:13.716941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:13.717338 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:13.717724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:13.718612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:13.726886 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:13.915806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1269/2507] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:13 2022] POST /identity/v3/projects => generated 365 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:13.919589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:13.940819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14dd25-0ba5-412c-a051-62d0c3b22d7f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.021445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.022346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.022736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.023152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.023985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.027262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.195830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1239/2508] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:13 2022] POST /identity/v3/users => generated 289 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:14.199965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:14.215630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e16cb-fbf4-4a7c-a267-daf7d75192da admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.222753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.223642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.224897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.225294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.226027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.227782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.247865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1270/2509] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:14.252101 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:14.262527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e3b7d-0dda-4181-a8b3-91df77db868d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.267916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.268807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.269102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.269381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] PATH_INFO: `/v3/projects/adaba04dc6ab4820b48b3b566c2e6702/users/4c3e9562059a4f529cfb74204c828078/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.286103 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adaba04dc6ab4820b48b3b566c2e6702, user_id=4c3e9562059a4f529cfb74204c828078, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.288933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.484976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1240/2510] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:14 2022] PUT /identity/v3/projects/adaba04dc6ab4820b48b3b566c2e6702/users/4c3e9562059a4f529cfb74204c828078/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:14.488889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:14.498879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dfa5e-6b3d-406a-a8a2-288d9ce5f8f8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.563943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.565244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.565536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.565818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.566807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.568437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772936c-7d7f-4579-ae8a-c0f5b7bde7cb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.583231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1271/2511] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:14.584224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:14.595085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148017d-6472-4849-88d3-fe8e0eb1a2e4 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.603776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.603776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.603776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.603776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] PATH_INFO: `/v3/projects/adaba04dc6ab4820b48b3b566c2e6702/users/4c3e9562059a4f529cfb74204c828078/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.604343 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adaba04dc6ab4820b48b3b566c2e6702, user_id=4c3e9562059a4f529cfb74204c828078, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.606576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.617367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1241/2512] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:14 2022] PUT /identity/v3/projects/adaba04dc6ab4820b48b3b566c2e6702/users/4c3e9562059a4f529cfb74204c828078/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:14.628314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.628772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.629189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.662899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] MFA Rules not processed for user `4c3e9562059a4f529cfb74204c828078`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:14.664275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:14.740508 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1272/2513] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:14.754454 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:14.756812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:14.819239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b43a0-5dc4-4439-8525-8eace40201a2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:14.823202 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:14.823891 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:14.824190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:14.824839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:14.835096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:14.844629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202c790-ff37-4b36-b587-a4adacb3bdf1 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:14.846062 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1242/2514] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:15.259017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:15.271440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c422-be4e-4226-bd36-968acd775ba5 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:15.281442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:15.282168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:15.282463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:15.282744 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:15.352653 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:15.354125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:15.364895 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1273/2515] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:15.690969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:15.961989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7e680-ef0c-49d3-9717-2fc3a19e71fd None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:15.963227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7e680-ef0c-49d3-9717-2fc3a19e71fd None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:15.963869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7e680-ef0c-49d3-9717-2fc3a19e71fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:15.998568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-86e7e680-ef0c-49d3-9717-2fc3a19e71fd None None] MFA Rules not processed for user `4df05cbf303c4524a665b429f240a17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:16.001533 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-86e7e680-ef0c-49d3-9717-2fc3a19e71fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:16.071405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1243/2516] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:16.988193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:19.993533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:20.018182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:20.090716 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2771e-720a-440f-9b93-e4a09a2a4d79 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:20.096106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:20.097072 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:20.097442 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:20.097803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] PATH_INFO: `/v3/users/901f81c893aa40de9598bc9676e52028` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:20.099132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901f81c893aa40de9598bc9676e52028)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:20.101381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:20.391565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901f81c893aa40de9598bc9676e52028'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:20.392984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901f81c893aa40de9598bc9676e52028'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:20.400649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901f81c893aa40de9598bc9676e52028'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:20.553691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901f81c893aa40de9598bc9676e52028'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:20.574451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901f81c893aa40de9598bc9676e52028'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:20.583856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1274/2517] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:20 2022] DELETE /identity/v3/users/901f81c893aa40de9598bc9676e52028 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:20.591073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:20.591652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:20.592182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:20.636671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:20.649320 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:20.650606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:20.781378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1244/2518] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:20.799532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:20.801602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:20.867877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5fce8-9a43-44fa-b1ba-a61d515d1565 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:20.872408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:20.873294 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:20.873660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:20.874472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:20.887413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:20.888946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:20.902903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1275/2519] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:21.698085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:21.699782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:21.721237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a1c46-4d68-4d50-addd-b957e06d878d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:21.726375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:21.727915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:21.728401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:21.729780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] PATH_INFO: `/v3/projects/79f4863513bb439ea18976c9953d065e` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:21.730153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f4863513bb439ea18976c9953d065e)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:21.732450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:21.893940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f4863513bb439ea18976c9953d065e', 'request_id': 'req-a322f644-46be-4d54-be52-fb083cc9c5af'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:21.900213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f4863513bb439ea18976c9953d065e', 'request_id': 'req-a322f644-46be-4d54-be52-fb083cc9c5af'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.014294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f4863513bb439ea18976c9953d065e', 'request_id': 'req-a322f644-46be-4d54-be52-fb083cc9c5af'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.046869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] The token cache is being invalidate because project 79f4863513bb439ea18976c9953d065e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:22.047436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.049014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.052266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1245/2520] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:21 2022] DELETE /identity/v3/projects/79f4863513bb439ea18976c9953d065e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:22.056189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:22.068505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9154-6729-43eb-8fcc-2433b656a43b service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:22.141572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:22.142514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:22.142804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:22.143136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] PATH_INFO: `/v3/users/4c3e9562059a4f529cfb74204c828078` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:22.144622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3e9562059a4f529cfb74204c828078)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:22.146771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:22.620630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3e9562059a4f529cfb74204c828078'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.621883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3e9562059a4f529cfb74204c828078'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.632610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3e9562059a4f529cfb74204c828078'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.736720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3e9562059a4f529cfb74204c828078'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.761642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3e9562059a4f529cfb74204c828078'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:22.772839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1276/2521] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:22 2022] DELETE /identity/v3/users/4c3e9562059a4f529cfb74204c828078 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:22.959301 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:23.168722 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:23.172589 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:23.192885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322f644-46be-4d54-be52-fb083cc9c5af admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:23.254234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:23.255168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:23.255451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:23.255735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] PATH_INFO: `/v3/users/88446ec114864875a88a20d634b4423a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:23.256952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88446ec114864875a88a20d634b4423a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:23.259301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:23.493874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88446ec114864875a88a20d634b4423a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.494663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88446ec114864875a88a20d634b4423a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.505659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88446ec114864875a88a20d634b4423a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.612153 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:23.617472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:23.639060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88446ec114864875a88a20d634b4423a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.640232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a289cc0-c680-4784-8d05-57564b653d5e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:23.645662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:23.646876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:23.647700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:23.648217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] PATH_INFO: `/v3/projects/adaba04dc6ab4820b48b3b566c2e6702` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:23.649758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=adaba04dc6ab4820b48b3b566c2e6702)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:23.654251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:23.656790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-fc08d81c-a2bd-4460-a133-3ec1800e2d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88446ec114864875a88a20d634b4423a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.666959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1246/2522] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:23 2022] DELETE /identity/v3/users/88446ec114864875a88a20d634b4423a => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:23.823815 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adaba04dc6ab4820b48b3b566c2e6702', 'request_id': 'req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.831027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adaba04dc6ab4820b48b3b566c2e6702', 'request_id': 'req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.862890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:23.953555 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adaba04dc6ab4820b48b3b566c2e6702', 'request_id': 'req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.966735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] The token cache is being invalidate because project adaba04dc6ab4820b48b3b566c2e6702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:23.967187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.967824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:23.969383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1277/2523] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:23 2022] DELETE /identity/v3/projects/adaba04dc6ab4820b48b3b566c2e6702 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:23.974706 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64b63-7268-4575-8b4e-ba713ce28b8c None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:23.975076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64b63-7268-4575-8b4e-ba713ce28b8c None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:23.975360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64b63-7268-4575-8b4e-ba713ce28b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:23.994494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eca64b63-7268-4575-8b4e-ba713ce28b8c None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:23.995663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eca64b63-7268-4575-8b4e-ba713ce28b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.057244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1247/2524] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:24.064412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.067149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.069040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.069395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.069668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.082510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5592c2d-30a9-436c-b6c8-64d0cac57dfb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.088274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.089625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.090290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.090808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] PATH_INFO: `/v3/projects/2ab754ce6b844222af7ddb23f93aa648` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.092320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab754ce6b844222af7ddb23f93aa648)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.096258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.107026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:24.108269 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.118538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1248/2525] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:24.122675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.133177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a980f-da14-48b8-a407-dfd707444497 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.137915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.139650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.139650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.139650 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.139978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:50:24.140291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.142478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.155274 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1249/2526] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:50:24.159953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.170869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b8ece-4f1b-4f3b-af55-aac8b2673da0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.179116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.179116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.179116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.179116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.179494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.180297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.218163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab754ce6b844222af7ddb23f93aa648', 'request_id': 'req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.224032 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab754ce6b844222af7ddb23f93aa648', 'request_id': 'req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.353671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1250/2527] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:24 2022] POST /identity/v3/projects => generated 401 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:24.356689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.377327 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab754ce6b844222af7ddb23f93aa648', 'request_id': 'req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.391078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] The token cache is being invalidate because project 2ab754ce6b844222af7ddb23f93aa648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:24.392967 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.394197 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.396541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1278/2528] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:24 2022] DELETE /identity/v3/projects/2ab754ce6b844222af7ddb23f93aa648 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:24.400464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.403263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.426351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a13d0-f6c6-495b-90af-8963908c0ebb admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.431512 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.432197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.432536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.432826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.433524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.435883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.487510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb0ea5-903b-4a17-9f89-8cc4a22e8092 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.492356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.493247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.493531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.493807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] PATH_INFO: `/v3/users/c0968fcc8f324700910ab82cc4d1eefe` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.495189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0968fcc8f324700910ab82cc4d1eefe)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.497211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.575687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1251/2529] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:24 2022] POST /identity/v3/users => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:24.579812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.581453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.593088 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c75bf-5fd5-42ec-8c7f-56fe46cfb645 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.597653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.598292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.598579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.598869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.599616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.601192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.612136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1252/2530] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:24.617463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.630711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b11ae-2357-47a4-9108-c3ca9f4e05d6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.635597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.636324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.636620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.636968 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.637751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.639433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.650566 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1253/2531] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:24.654590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.664001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570221b6-de88-4b60-80d2-4df684b70565 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.668707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.669571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.669851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.670126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] PATH_INFO: `/v3/projects/45ff770fdca64e539f62153f07db0b23/users/3348a983e88c4036a42974a25f16d33d/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.684552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ff770fdca64e539f62153f07db0b23, user_id=3348a983e88c4036a42974a25f16d33d, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.687176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57061c27-24a7-49f2-9de2-1213857636cf admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:24.811057 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1254/2532] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:24 2022] PUT /identity/v3/projects/45ff770fdca64e539f62153f07db0b23/users/3348a983e88c4036a42974a25f16d33d/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:24.820085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0968fcc8f324700910ab82cc4d1eefe'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.821200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0968fcc8f324700910ab82cc4d1eefe'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.824930 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596b51-e408-4726-9382-afe70a7768ef None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.825229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596b51-e408-4726-9382-afe70a7768ef None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.825583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596b51-e408-4726-9382-afe70a7768ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.829605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0968fcc8f324700910ab82cc4d1eefe'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.853852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-30596b51-e408-4726-9382-afe70a7768ef None None] MFA Rules not processed for user `3348a983e88c4036a42974a25f16d33d`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:24.857889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-30596b51-e408-4726-9382-afe70a7768ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.919078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1255/2533] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:24.922998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30596b51-e408-4726-9382-afe70a7768ef None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:24.924921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-30596b51-e408-4726-9382-afe70a7768ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:24.987771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0968fcc8f324700910ab82cc4d1eefe'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:24.988006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30596b51-e408-4726-9382-afe70a7768ef None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:24.992964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:24.993623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:24.993910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:24.994193 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:24.994993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:24.997006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.000817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0968fcc8f324700910ab82cc4d1eefe'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.007062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1279/2534] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:24 2022] DELETE /identity/v3/users/c0968fcc8f324700910ab82cc4d1eefe => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:25.120319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1256/2535] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:24 2022] POST /identity/v3/projects => generated 399 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:25.124319 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.187632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db4c3-6243-499a-8d36-c5e445c53bd9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.243231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.243914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.244202 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.244483 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.245172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.247122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.263191 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.265075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:25.281098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159f13-4bb6-4949-83bc-97b60d229e8c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.287241 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.288134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.289762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.290102 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] PATH_INFO: `/v3/projects/fb3439d6822b4cb1af0fffd7c7943716` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.291356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3439d6822b4cb1af0fffd7c7943716)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.306478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.410481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1280/2536] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:25 2022] POST /identity/v3/users => generated 306 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:25.414271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.425141 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531a752-52b8-4beb-83db-eee6afc24ee3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.430015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.430690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.431011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.431342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.432079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.433689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.449597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1281/2537] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:25.453696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.459545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3439d6822b4cb1af0fffd7c7943716', 'request_id': 'req-158aa4c3-0457-4bd3-bed1-1faa402230fe'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.464097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71389e-5585-4fba-b859-657bcdfb2752 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.467047 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3439d6822b4cb1af0fffd7c7943716', 'request_id': 'req-158aa4c3-0457-4bd3-bed1-1faa402230fe'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.522187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.523129 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.523420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.523698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] PATH_INFO: `/v3/projects/bcaf1784d7e343718f03125e5111ca0c/users/d4c5b3b7c4804ecd89144e8c3bf3da25/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.540011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcaf1784d7e343718f03125e5111ca0c, user_id=d4c5b3b7c4804ecd89144e8c3bf3da25, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.542619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.581835 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3439d6822b4cb1af0fffd7c7943716', 'request_id': 'req-158aa4c3-0457-4bd3-bed1-1faa402230fe'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.594030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] The token cache is being invalidate because project fb3439d6822b4cb1af0fffd7c7943716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:25.594319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.595627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:25.597175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1257/2538] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:25 2022] DELETE /identity/v3/projects/fb3439d6822b4cb1af0fffd7c7943716 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:25.663261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1282/2539] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:25 2022] PUT /identity/v3/projects/bcaf1784d7e343718f03125e5111ca0c/users/d4c5b3b7c4804ecd89144e8c3bf3da25/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:25.667426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.668986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:25.682845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa4c3-0457-4bd3-bed1-1faa402230fe admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.745531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.746224 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.746516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.746797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.747671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.749494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca44e3-b78c-4e53-ade4-35d45fa9104d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.760681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1258/2540] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:25.765089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.776631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0625-4597-476d-8eac-e83bfcf78abe admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:25.781806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:25.783271 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.783556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.783837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] PATH_INFO: `/v3/projects/bcaf1784d7e343718f03125e5111ca0c/users/d4c5b3b7c4804ecd89144e8c3bf3da25/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.785888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcaf1784d7e343718f03125e5111ca0c, user_id=d4c5b3b7c4804ecd89144e8c3bf3da25, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:25.788589 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:25.800613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1283/2541] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:25 2022] PUT /identity/v3/projects/bcaf1784d7e343718f03125e5111ca0c/users/d4c5b3b7c4804ecd89144e8c3bf3da25/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:25.811640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:25.812007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:25.812290 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:25.843687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] MFA Rules not processed for user `d4c5b3b7c4804ecd89144e8c3bf3da25`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:25.845335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:25.911734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1259/2542] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:25.922337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:25.924705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:25.989889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568d9f-cd1b-496d-a6a0-c9c16ab167e2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:26.002747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:26.002747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:26.002747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:26.002747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:26.006770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:26.008312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:26.018976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1284/2543] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:25 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:26.503831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:26.513120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ed91f-7268-446f-9c1c-65d2f9b9e74e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:26.516707 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:26.517864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:26.518148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:26.518425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:26.520182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:26.580513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:26.582309 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa086334-3d0f-4e41-9587-0f20571fe524 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:26.591808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1260/2544] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:27.827131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:28.798851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:32.204004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:32.676876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:32.745897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4339-5629-4b28-9f60-85495df7360f service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:32.752673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:32.757081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:32.757081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:32.757081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] PATH_INFO: `/v3/users/3348a983e88c4036a42974a25f16d33d` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:32.757081 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3348a983e88c4036a42974a25f16d33d)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:32.757915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:32.975881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:33.089949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3348a983e88c4036a42974a25f16d33d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:33.090628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3348a983e88c4036a42974a25f16d33d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:33.098807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3348a983e88c4036a42974a25f16d33d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:33.244792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3348a983e88c4036a42974a25f16d33d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:33.257162 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3348a983e88c4036a42974a25f16d33d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:33.263234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1285/2545] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:32 2022] DELETE /identity/v3/users/3348a983e88c4036a42974a25f16d33d => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:33.267919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:33.268403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:33.268689 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:33.288176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:33.289685 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:33.353231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1261/2546] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:33.364836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:33.366290 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:33.428331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c78e2d-b181-4ad5-b04c-1efa732a94a2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:33.431938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:33.432597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:33.436989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:33.436989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:33.442839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:33.444300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:33.454685 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1286/2547] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:34.023832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:34.023832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:34.055663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f0dc-7c02-47af-883b-2a9d68a100ab None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:34.062461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:34.064055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:34.064330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:34.064613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] PATH_INFO: `/v3/projects/45ff770fdca64e539f62153f07db0b23` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:34.066019 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ff770fdca64e539f62153f07db0b23)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:34.068271 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:34.270762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ff770fdca64e539f62153f07db0b23', 'request_id': 'req-3e3720ec-e629-4d93-adbe-97b295202b81'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.277308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ff770fdca64e539f62153f07db0b23', 'request_id': 'req-3e3720ec-e629-4d93-adbe-97b295202b81'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.358880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ff770fdca64e539f62153f07db0b23', 'request_id': 'req-3e3720ec-e629-4d93-adbe-97b295202b81'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.374627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] The token cache is being invalidate because project 45ff770fdca64e539f62153f07db0b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:34.375278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.376183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.378616 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1262/2548] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:34 2022] DELETE /identity/v3/projects/45ff770fdca64e539f62153f07db0b23 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:34.382641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:34.399946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf298-14d8-42f7-abda-2c883547b006 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:34.460703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:34.466318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:34.466318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:34.466318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] PATH_INFO: `/v3/users/d4c5b3b7c4804ecd89144e8c3bf3da25` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:34.466318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c5b3b7c4804ecd89144e8c3bf3da25)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:34.466318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:34.679687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c5b3b7c4804ecd89144e8c3bf3da25'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.680364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c5b3b7c4804ecd89144e8c3bf3da25'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.688736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c5b3b7c4804ecd89144e8c3bf3da25'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.862104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c5b3b7c4804ecd89144e8c3bf3da25'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.875132 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1bfb905a-7b1b-479f-a9ff-b182227f0325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c5b3b7c4804ecd89144e8c3bf3da25'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:34.881090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1287/2549] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:34 2022] DELETE /identity/v3/users/d4c5b3b7c4804ecd89144e8c3bf3da25 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:35.360670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:35.422149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:35.423865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:35.446077 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3720ec-e629-4d93-adbe-97b295202b81 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:35.504581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:35.505628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:35.506108 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:35.506384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] PATH_INFO: `/v3/projects/bcaf1784d7e343718f03125e5111ca0c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:35.508153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcaf1784d7e343718f03125e5111ca0c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:35.510306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:35.678614 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:35.695399 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcaf1784d7e343718f03125e5111ca0c', 'request_id': 'req-0b55d033-0e8f-48e0-9839-5eb28c470f9a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:35.700867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcaf1784d7e343718f03125e5111ca0c', 'request_id': 'req-0b55d033-0e8f-48e0-9839-5eb28c470f9a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:35.803498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcaf1784d7e343718f03125e5111ca0c', 'request_id': 'req-0b55d033-0e8f-48e0-9839-5eb28c470f9a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:35.815050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] The token cache is being invalidate because project bcaf1784d7e343718f03125e5111ca0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:35.815342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:35.815972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0b55d033-0e8f-48e0-9839-5eb28c470f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:35.817770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1263/2550] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:35 2022] DELETE /identity/v3/projects/bcaf1784d7e343718f03125e5111ca0c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:35.822989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:35.823347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:35.823784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:35.843859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:35.845726 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:35.902769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1288/2551] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:35.911536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:35.911831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:35.912140 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:35.931720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:35.932938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:35.941799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1264/2552] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:35.945969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:35.955772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd933b-d9ef-448b-ab37-ca60b09d1fff None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:35.960507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:35.961151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:35.961434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:35.961707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:35.962518 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:50:35.962765 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:35.964413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:35.975874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1289/2553] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:50:35.982150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:35.991801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157d2e3-f364-412c-9834-62a57c974929 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:35.996534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:35.997179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:35.997459 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:35.997738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:35.998438 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.000359 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.177295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1265/2554] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:35 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:36.181501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.192277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615875e-479a-4aa8-a164-5fa2f0601a2a admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:36.246309 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:36.247116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.247403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.247677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.248418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.250312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.399005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1290/2555] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:36 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:36.403215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.413332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70402953-d51a-41d3-81ba-cadd248be356 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:36.418283 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:36.418939 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.419239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.419517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.420381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.422026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.432694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1266/2556] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:36.436698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.447359 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238bc8f-f219-4472-ade5-6cdf647b6b71 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:36.451882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:36.452550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.452835 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.453110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.454412 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.456038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e3997-e43d-45b6-8305-27d3c14560ca admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.466229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1291/2557] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:36.470372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.480076 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba13acb-4829-4ec9-a43d-2a754e3f8b58 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:36.484563 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:36.485931 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.486209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.486482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] PATH_INFO: `/v3/projects/d09993bdf426495d9c4f6e759de9f5b4/users/01305555bdfe4938941532796b60678a/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.501236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09993bdf426495d9c4f6e759de9f5b4, user_id=01305555bdfe4938941532796b60678a, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.503934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.610263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1267/2558] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:36 2022] PUT /identity/v3/projects/d09993bdf426495d9c4f6e759de9f5b4/users/01305555bdfe4938941532796b60678a/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:36.623756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.624172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.624597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.654286 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] MFA Rules not processed for user `01305555bdfe4938941532796b60678a`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:36.657585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:36.723476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1292/2559] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:36.728745 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.797145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c95d7-eec3-4ef8-9955-67919e1f4406 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:36.801858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:36.802546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:36.802830 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:36.803131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:36.803857 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:36.805751 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:36.952235 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:36.953648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:36.959815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1268/2560] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:36 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:36.963688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.012180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7b2b8-1dd3-4032-b1e9-61e78c2a1c48 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.031046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44672599-eca7-4396-a44c-1acd0e71f900 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.039826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.040505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.040801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.041170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.041776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.043846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.065119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.065812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.066104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.066640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.068465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:37.130189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.131657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.141204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1293/2561] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:50:37.210701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1269/2562] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:36 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:37.214066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.224474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151462a-74a1-4459-9186-d812f90741c2 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.229103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.229766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.230057 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.230340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.231102 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.232693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.243118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1294/2563] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:37.247092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.256993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90047e9-64b2-460c-880b-545e60f19a43 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.261652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.262523 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.262812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.263123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] PATH_INFO: `/v3/projects/22f08eed70784e74b278330cf0227537/users/d31b95d2acf44f3f89f62b1ce1e402eb/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.278393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f08eed70784e74b278330cf0227537, user_id=d31b95d2acf44f3f89f62b1ce1e402eb, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.281098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.421833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1270/2564] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:37 2022] PUT /identity/v3/projects/22f08eed70784e74b278330cf0227537/users/d31b95d2acf44f3f89f62b1ce1e402eb/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:37.426161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.436474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a362-1530-4336-9888-d8c865a34e58 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.493646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.494334 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.494626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.494904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.495679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.499196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14ac60-865a-4266-9919-05468cd7dc1a admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.510779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1295/2565] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:37.512002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.522322 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1c8e3-01e5-4265-93f6-298150b399a0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.526994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.527876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.528155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.528426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] PATH_INFO: `/v3/projects/22f08eed70784e74b278330cf0227537/users/d31b95d2acf44f3f89f62b1ce1e402eb/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.530368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f08eed70784e74b278330cf0227537, user_id=d31b95d2acf44f3f89f62b1ce1e402eb, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.533085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.543776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1271/2566] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:37 2022] PUT /identity/v3/projects/22f08eed70784e74b278330cf0227537/users/d31b95d2acf44f3f89f62b1ce1e402eb/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:37.560097 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.560419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.560710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.590174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] MFA Rules not processed for user `d31b95d2acf44f3f89f62b1ce1e402eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:37.591530 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:37.661066 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1296/2567] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:37.673134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:37.674883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:37.734448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597b783-039e-488c-815c-3291a0a5bed0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:37.746296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:37.746296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:37.746296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:37.746296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:37.750149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:37.751712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e7dd6-fdb8-43fe-9bb5-aabccdaf9142 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:37.761385 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1272/2568] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:37.890092 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:38.166214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:38.176887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded49eb-16c2-41bc-bdcf-91283f692f94 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:38.180694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:38.181354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:38.181647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:38.181943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:38.241898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:38.243396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:38.252575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1297/2569] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:50:38.456183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:41.427191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:41.975254 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:44.244029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:45.549403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:45.549927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:45.550030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:45.568444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:45.569665 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:45.630250 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1273/2570] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:45.642397 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:45.643958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:45.718111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5002fb3-9464-4c86-9209-1e4cb84ab667 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:45.723374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:45.724042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:45.724332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:45.724612 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:45.733112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:45.734681 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f75718-fb82-40f4-9a5f-01a1815a070b service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:45.744065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1298/2571] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:50:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:50:46.038567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:46.447123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:46.453396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:46.525914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e9a7f-732d-453c-a2ab-f7f027895f42 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:46.530916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:46.531821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:46.532131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:46.532975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] PATH_INFO: `/v3/users/01305555bdfe4938941532796b60678a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:46.536124 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01305555bdfe4938941532796b60678a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:46.536519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:46.796450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01305555bdfe4938941532796b60678a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:46.797658 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01305555bdfe4938941532796b60678a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:46.805718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01305555bdfe4938941532796b60678a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:46.894741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01305555bdfe4938941532796b60678a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:46.908794 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01305555bdfe4938941532796b60678a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:46.917971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1274/2572] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:46 2022] DELETE /identity/v3/users/01305555bdfe4938941532796b60678a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:46.919742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:46.920226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:46.920358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:46.939439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:46.940727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:47.014587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1299/2573] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:47.021839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:47.024748 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:47.077938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:47.102982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc053-21a3-48a3-bb1c-77a41a3bdd6c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:47.107236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:47.108147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:47.108664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:47.109156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:47.128106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:47.130671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:47.150747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1275/2574] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:47.936976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:47.938594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:47.956617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11feba7b-6581-4318-87c5-2ea393bc5aac None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:47.961668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:47.963355 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:47.967779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:47.967779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] PATH_INFO: `/v3/projects/d09993bdf426495d9c4f6e759de9f5b4` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:47.967779 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09993bdf426495d9c4f6e759de9f5b4)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:47.972247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:48.328730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09993bdf426495d9c4f6e759de9f5b4', 'request_id': 'req-98fb44e4-3662-4ce6-ad73-f565dd682e12'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:48.339951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09993bdf426495d9c4f6e759de9f5b4', 'request_id': 'req-98fb44e4-3662-4ce6-ad73-f565dd682e12'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:48.455756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09993bdf426495d9c4f6e759de9f5b4', 'request_id': 'req-98fb44e4-3662-4ce6-ad73-f565dd682e12'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:48.468810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] The token cache is being invalidate because project d09993bdf426495d9c4f6e759de9f5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:48.469184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:48.479839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:48.479839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1300/2575] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:47 2022] DELETE /identity/v3/projects/d09993bdf426495d9c4f6e759de9f5b4 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:48.481614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:48.493400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fd936-aa3e-4f30-a703-18c802960931 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:48.522120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:48.559617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:48.560610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:48.560966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:48.561453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] PATH_INFO: `/v3/users/d31b95d2acf44f3f89f62b1ce1e402eb` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:48.562797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d31b95d2acf44f3f89f62b1ce1e402eb)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:48.565568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:49.268815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd31b95d2acf44f3f89f62b1ce1e402eb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:49.269469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd31b95d2acf44f3f89f62b1ce1e402eb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:49.276452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd31b95d2acf44f3f89f62b1ce1e402eb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:49.386636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd31b95d2acf44f3f89f62b1ce1e402eb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:49.400592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-4471d12c-bfd0-42fc-a931-a9272c07a0c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd31b95d2acf44f3f89f62b1ce1e402eb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:49.411820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1276/2576] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:48 2022] DELETE /identity/v3/users/d31b95d2acf44f3f89f62b1ce1e402eb => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:50.096759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:50.098256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:50.122868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb44e4-3662-4ce6-ad73-f565dd682e12 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:50.211259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:50.212219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.212528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.212912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] PATH_INFO: `/v3/projects/22f08eed70784e74b278330cf0227537` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.214264 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f08eed70784e74b278330cf0227537)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:50.216448 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:50.355802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f08eed70784e74b278330cf0227537', 'request_id': 'req-27a2fa86-7c96-46c8-8166-e5be2576cd1c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:50.362037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f08eed70784e74b278330cf0227537', 'request_id': 'req-27a2fa86-7c96-46c8-8166-e5be2576cd1c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:50.443323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f08eed70784e74b278330cf0227537', 'request_id': 'req-27a2fa86-7c96-46c8-8166-e5be2576cd1c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:50.457182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] The token cache is being invalidate because project 22f08eed70784e74b278330cf0227537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:50.457539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:50.458213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-27a2fa86-7c96-46c8-8166-e5be2576cd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:50.463491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1301/2577] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:50 2022] DELETE /identity/v3/projects/22f08eed70784e74b278330cf0227537 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:50.465306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.466466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.467204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.489491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:50.492505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:50.574211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1277/2578] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:50.576190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.576477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.576838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.585492 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:50.599947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:50.601178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:50.610201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1302/2579] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:50.614361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:50.627150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319db5-56b9-4c80-9d02-f1d7a3664ec5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:50.635797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:50.638008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.638008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.638008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.638008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:50:50.638341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:50.640026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:50.657356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1278/2580] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:50:50.658362 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:50.670034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07bc34-d297-4044-9dda-cb85d825c615 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:50.676640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:50.677320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.677606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.677883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.678578 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:50.680541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:50.838428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1303/2581] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:50 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:50:50.842484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:50.854125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1af2-51f4-4e59-a38e-f5e7986caaa7 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:50.929045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:50.960159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:50.961249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:50.961741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:50.962208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:50.962911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:50.965375 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.119338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1279/2582] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:50 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:51.123626 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.137912 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76722c0-f4fb-45af-85b2-03e001d820f5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.144520 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.145461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.145774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.146053 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.146816 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.149360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.176235 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.177255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1304/2583] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:51.200208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e62a75-da1b-4baa-bd96-187bcb6f0f7e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.205804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.206587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.207021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.207423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.208255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.209969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca2861-dede-43b2-9edb-7b368760ba50 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.225825 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1280/2584] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:50:51.230065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.241573 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784ce8c-cf4c-4e0a-b08e-2be3848c2792 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.246724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.247675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.248005 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.248288 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] PATH_INFO: `/v3/projects/a5dc347b400b43128da878b038e738fd/users/e2a745597220407da7c3f809e9e38b81/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.264242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5dc347b400b43128da878b038e738fd, user_id=e2a745597220407da7c3f809e9e38b81, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.266970 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.396904 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1305/2585] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:51 2022] PUT /identity/v3/projects/a5dc347b400b43128da878b038e738fd/users/e2a745597220407da7c3f809e9e38b81/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:51.410044 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.410348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.410631 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.443613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] MFA Rules not processed for user `e2a745597220407da7c3f809e9e38b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:51.444860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:51.528139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1281/2586] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:51.532783 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.606624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d650d19-e41d-41ea-9fe0-0cf41fbb190c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.612367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.613211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.613600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.613801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.614440 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.616477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.736711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.742280 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1306/2587] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:50:51 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:51.748517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17926a20-3263-4829-b1fb-e275bc60e452 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.809805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.810521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.810846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.811153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.811861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.814425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.935793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1282/2588] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:50:51 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:50:51.945019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.955859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39300c32-18c2-47b9-bac4-47d508e0c147 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:51.962670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:51.965723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:51.965723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:51.965723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:51.965723 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:51.967509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:51.983676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1307/2589] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:51.988189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:51.998579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc966846-20a8-4ddc-87af-90d881e21332 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:52.003578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:52.004560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:52.004910 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:52.005070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] PATH_INFO: `/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee/users/37225085abd0499f97599fddba609a86/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:52.020999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1571beb7fe464ecaab1f92e9b0cadcee, user_id=37225085abd0499f97599fddba609a86, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:52.023756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:52.591586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1283/2590] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:51 2022] PUT /identity/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee/users/37225085abd0499f97599fddba609a86/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:52.595905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:52.605968 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a159b-8b9d-429c-8fed-bdee72c17490 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:52.661437 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:52.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:52.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:52.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:52.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:52.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447f56b-e7bb-4f64-950c-0c8260ccd45b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:52.675220 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1308/2591] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:50:52.679432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:52.690471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddac4c-d82f-4238-93cb-fa0d38dc510a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:52.696418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:52.697303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:52.697590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:52.697872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] PATH_INFO: `/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee/users/37225085abd0499f97599fddba609a86/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:52.699935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1571beb7fe464ecaab1f92e9b0cadcee, user_id=37225085abd0499f97599fddba609a86, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:52.702540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:52.713626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1284/2592] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:50:52 2022] PUT /identity/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee/users/37225085abd0499f97599fddba609a86/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:52.735935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:52.736234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:52.736522 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:52.763706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] MFA Rules not processed for user `37225085abd0499f97599fddba609a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:52.764919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:52.804206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:52.848966 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1309/2593] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:50:52.856323 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:52.861728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:52.932480 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:52.939272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251f0c-025e-4d9b-8d36-f570627f74c8 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:52.943671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:52.944495 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:52.944802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:52.945200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:52.955427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:52.957002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f438cd-c3cb-4888-b940-114b2de89d56 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:52.967673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1285/2594] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:50:53.634243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:53.713209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bff3e-1d58-4bcd-b4c5-9757de936a5a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:53.718490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:53.720076 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:53.720363 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:53.720640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] PATH_INFO: `/v3/users/e2a745597220407da7c3f809e9e38b81` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:53.721841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a745597220407da7c3f809e9e38b81)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:53.723803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:53.959386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a745597220407da7c3f809e9e38b81'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:53.960065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a745597220407da7c3f809e9e38b81'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:53.968446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a745597220407da7c3f809e9e38b81'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:54.106444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a745597220407da7c3f809e9e38b81'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:54.122804 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a745597220407da7c3f809e9e38b81'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:54.133687 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1310/2595] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:53 2022] DELETE /identity/v3/users/e2a745597220407da7c3f809e9e38b81 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:54.135293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:54.136164 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:54.136458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:54.154647 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:50:54.156199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:54.233370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1286/2596] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:50:54.243300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:54.244883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:54.269340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:54.315542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c54a0-7891-42f3-bfac-c7e322563e46 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:54.320919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:54.321642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:54.322001 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:54.324927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:54.337909 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:54.339575 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:54.351268 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1311/2597] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:50:54.986845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:54.986845 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:55.005079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fde7d-dfc6-4352-95a8-c537d2b08654 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:55.013424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:55.014306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:55.014593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:55.015085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] PATH_INFO: `/v3/projects/a5dc347b400b43128da878b038e738fd` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:55.016298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5dc347b400b43128da878b038e738fd)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:55.018212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:55.180660 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5dc347b400b43128da878b038e738fd', 'request_id': 'req-63cd6901-0a5d-4f99-b1e2-921808c2940d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.190759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5dc347b400b43128da878b038e738fd', 'request_id': 'req-63cd6901-0a5d-4f99-b1e2-921808c2940d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.357407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5dc347b400b43128da878b038e738fd', 'request_id': 'req-63cd6901-0a5d-4f99-b1e2-921808c2940d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.369916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] The token cache is being invalidate because project a5dc347b400b43128da878b038e738fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:55.370208 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.370874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.372673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1287/2598] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:54 2022] DELETE /identity/v3/projects/a5dc347b400b43128da878b038e738fd => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:50:55.376528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:55.388084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0361b-9c25-4d06-92a7-181cba1231af service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:55.451047 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:55.451931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:55.452213 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:55.452488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] PATH_INFO: `/v3/users/37225085abd0499f97599fddba609a86` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:55.454193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37225085abd0499f97599fddba609a86)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:55.456134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:55.959628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37225085abd0499f97599fddba609a86'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.960201 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37225085abd0499f97599fddba609a86'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:55.967031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37225085abd0499f97599fddba609a86'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:56.117416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37225085abd0499f97599fddba609a86'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:56.134238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-a70cce2f-b486-4715-bfa6-337c3a32c8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37225085abd0499f97599fddba609a86'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:56.139791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1312/2599] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:50:55 2022] DELETE /identity/v3/users/37225085abd0499f97599fddba609a86 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:50:56.819969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:50:56.819969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:50:56.825970 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd6901-0a5d-4f99-b1e2-921808c2940d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:50:56.845014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:50:56.886039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:50:56.886906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:50:56.887227 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:50:56.887510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] PATH_INFO: `/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:50:56.888793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1571beb7fe464ecaab1f92e9b0cadcee)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:50:56.890885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:50:57.123042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1571beb7fe464ecaab1f92e9b0cadcee', 'request_id': 'req-01d953ba-b665-4a16-923d-cbf09241118a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:57.128906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1571beb7fe464ecaab1f92e9b0cadcee', 'request_id': 'req-01d953ba-b665-4a16-923d-cbf09241118a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:57.305890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1571beb7fe464ecaab1f92e9b0cadcee', 'request_id': 'req-01d953ba-b665-4a16-923d-cbf09241118a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:57.318190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] The token cache is being invalidate because project 1571beb7fe464ecaab1f92e9b0cadcee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:50:57.318481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:57.319138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-01d953ba-b665-4a16-923d-cbf09241118a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:50:57.320776 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1288/2600] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:50:56 2022] DELETE /identity/v3/projects/1571beb7fe464ecaab1f92e9b0cadcee => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:04.736706 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:04.737287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:04.737436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:04.739633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1313/2601] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:51:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:51:04.746280 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59be85-1181-4b6b-a32c-43df0abd3c30 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:04.747148 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59be85-1181-4b6b-a32c-43df0abd3c30 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:04.747315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59be85-1181-4b6b-a32c-43df0abd3c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:04.774972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fc59be85-1181-4b6b-a32c-43df0abd3c30 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:04.777022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fc59be85-1181-4b6b-a32c-43df0abd3c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:04.853965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1289/2602] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:04.865457 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:04.869766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:04.916474 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:04.934702 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f1749-c487-4391-b901-3de4bfcd1bb2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:04.938800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20bf88e1a4594d4f973c5dd049aeadd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:04.939491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:04.939773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:04.940051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:04.951182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:04.952615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057289f-a679-46a4-a146-c242dad9bc58 service glance] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:04.966420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1314/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:05.194879 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.195206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.195486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] PATH_INFO: `/` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.197030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1290/2604] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:51:05.202456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f5cc-b7cf-442d-925a-d5e44ae92e70 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.202829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f5cc-b7cf-442d-925a-d5e44ae92e70 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.203206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f5cc-b7cf-442d-925a-d5e44ae92e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.222596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a4d3f5cc-b7cf-442d-925a-d5e44ae92e70 None None] MFA Rules not processed for user `bf8b8605da7d49cfbdfb0162cbbf2ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:05.223843 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a4d3f5cc-b7cf-442d-925a-d5e44ae92e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:05.281127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1315/2605] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:05.293568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:05.295319 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:05.319037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea5178-5a12-487d-b431-9483f0871b5b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:05.323718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:05.324361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.324646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.324995 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.333318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:05.334908 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5847-32f5-4603-8182-2b7b4ab9f2df service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:05.342797 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1291/2606] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:05.432445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbb40-be43-4fe9-978a-2ae2ca378852 None None] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.432747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbb40-be43-4fe9-978a-2ae2ca378852 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.433030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbb40-be43-4fe9-978a-2ae2ca378852 None None] PATH_INFO: `/` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.434485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1316/2607] 10.222.0.36 () {54 vars in 921 bytes} [Thu Jun 16 19:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 19:51:05.440590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440416-39ee-44b3-8b96-dd6b06e2a650 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.441132 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440416-39ee-44b3-8b96-dd6b06e2a650 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.441414 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440416-39ee-44b3-8b96-dd6b06e2a650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.460420 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4a440416-39ee-44b3-8b96-dd6b06e2a650 None None] MFA Rules not processed for user `a82e215bee54480b8e4f69ca6352a865`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:05.461583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4a440416-39ee-44b3-8b96-dd6b06e2a650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:05.473301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1292/2608] 10.222.0.36 () {58 vars in 1018 bytes} [Thu Jun 16 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:05.484601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbb40-be43-4fe9-978a-2ae2ca378852 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:05.498179 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbb40-be43-4fe9-978a-2ae2ca378852 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:05.502387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:05.503085 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:05.503398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:05.503676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:05.514679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:05.516665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:05.528735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1317/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:08.005969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb82d46-a0c0-4fde-b5ba-2123bd1598f3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:08.006445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb82d46-a0c0-4fde-b5ba-2123bd1598f3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:08.006541 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb82d46-a0c0-4fde-b5ba-2123bd1598f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:08.023557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0eb82d46-a0c0-4fde-b5ba-2123bd1598f3 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:08.024725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0eb82d46-a0c0-4fde-b5ba-2123bd1598f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:08.041450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:08.080604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1293/2610] 10.222.0.36 () {58 vars in 1009 bytes} [Thu Jun 16 19:51:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:08.094713 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:08.096158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:08.171812 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56797-0b56-46e7-b806-2ba3226d213b service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:08.177426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:08.178078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:08.178371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:08.178656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:08.188367 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:08.189964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:08.199124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1318/2611] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:51:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:51:18.174215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=15 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:18.195256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=70 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:26.316248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:26.316726 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:26.316844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:26.334572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] MFA Rules not processed for user `1e5c2a70586143ac94e5be00608aa6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:26.335799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:26.343021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1294/2612] 10.222.0.36 () {58 vars in 967 bytes} [Thu Jun 16 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:26.364703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:26.375161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b64470-c3a5-4c08-816e-5efb2fe6cb98 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:26.380111 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e2bf373b15496597a6f18d14244495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:26.380784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:26.381069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:26.381348 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:26.389306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:26.391416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:26.399386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1319/2613] 10.222.0.36 () {60 vars in 1384 bytes} [Thu Jun 16 19:51:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:51:36.329561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:36.396217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:39.912780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:39.914184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:39.982058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31d9bd-028f-40b7-ba09-145f15b63387 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:39.987502 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:39.988139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:39.988424 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:39.988701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:39.990475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:40.002357 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:40.003965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:40.013718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1295/2614] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:41.855874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:41.865921 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd3d33-4d18-46c2-92e4-e27e412574d1 service swift] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:41.869625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:41.871075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:41.871075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:41.871075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:41.872622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:41.937563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:41.939061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:41.949381 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1320/2615] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:51:45.341027 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:45.342478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:45.356295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265bff-02c9-43a4-b74e-ac0486ee9bd4 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:45.362209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:45.362854 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:45.363171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:45.363451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:45.365750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:45.383008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:45.385498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:45.400056 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1296/2616] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:45.546531 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:45.548009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:45.560757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425abaf6-3c9b-402f-8434-6e9d92d1a74f service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:45.565802 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:45.566672 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:45.566978 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:45.567257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] PATH_INFO: `/v3/users/4df05cbf303c4524a665b429f240a17a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:45.568430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df05cbf303c4524a665b429f240a17a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:45.570335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:45.901259 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df05cbf303c4524a665b429f240a17a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:45.901910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df05cbf303c4524a665b429f240a17a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:45.909198 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df05cbf303c4524a665b429f240a17a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:45.995408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df05cbf303c4524a665b429f240a17a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.007821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-940ec495-6601-4c8a-bec4-62dcca1ee689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df05cbf303c4524a665b429f240a17a'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.013701 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1321/2617] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:51:45 2022] DELETE /identity/v3/users/4df05cbf303c4524a665b429f240a17a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:46.263746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:46.275232 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a7901-74c3-4bc6-af5c-e7e2f97f3a46 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:46.332753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:46.333520 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.333808 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.334089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] PATH_INFO: `/v3/projects/47a4ab888dd74218ab5d7812ad095371` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.335312 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a4ab888dd74218ab5d7812ad095371)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:46.337251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:46.464439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a4ab888dd74218ab5d7812ad095371', 'request_id': 'req-92c5fa18-9e2f-4c35-8623-4e2339befe22'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.469820 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a4ab888dd74218ab5d7812ad095371', 'request_id': 'req-92c5fa18-9e2f-4c35-8623-4e2339befe22'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.566759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a4ab888dd74218ab5d7812ad095371', 'request_id': 'req-92c5fa18-9e2f-4c35-8623-4e2339befe22'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.580683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] The token cache is being invalidate because project 47a4ab888dd74218ab5d7812ad095371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:51:46.580974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.581691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-92c5fa18-9e2f-4c35-8623-4e2339befe22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:46.583865 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1297/2618] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:51:46 2022] DELETE /identity/v3/projects/47a4ab888dd74218ab5d7812ad095371 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:46.588914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.589416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.589539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.608635 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:46.609881 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:46.671324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1322/2619] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:46.679730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.680092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.680452 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.694048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:46.699547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:46.700756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:46.709167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1298/2620] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:46.713564 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:46.724503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529effc-d03a-4ed0-a581-856210576c40 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:46.729721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:46.730371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.730651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.730981 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.731707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:51:46.732014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:46.733610 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:46.735108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:46.745780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1323/2621] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:51:46.750063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:46.759074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f08a18-c98f-4f8b-b51d-e5315256bc8d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:46.763734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:46.764870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.765152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.765425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.766116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:46.768079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:46.887757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1299/2622] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:51:46 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:51:46.892260 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:46.902306 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7c30-be39-4b07-8db9-7ba6badfe5c8 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:46.958724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:46.959421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:46.959712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:46.959989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:46.960680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:46.962579 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:47.062889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1324/2623] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:51:46 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:51:47.067296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:47.078188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bde6c-f1a1-4472-bbe6-8d40a0922367 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:47.082827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:47.083507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:47.083795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:47.084069 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:47.084822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:47.086445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:47.096231 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1300/2624] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:51:47.099943 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:47.109730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664156-ec3d-4b94-849a-53e57b5db94f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:47.114360 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:47.115050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:47.115341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:47.115622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:47.116349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:47.117934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e726a48-36a1-4896-9315-033e53f02a85 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:47.128123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1325/2625] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:51:47.132068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:47.142315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea707a-747f-4c98-8ffb-c1797d866678 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:47.146829 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:47.147735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:47.148022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:47.148307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] PATH_INFO: `/v3/projects/5dde443064694864b3c43f234e5a2fc0/users/2c5005a536c746fe8c86bc03426e6062/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:47.162695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dde443064694864b3c43f234e5a2fc0, user_id=2c5005a536c746fe8c86bc03426e6062, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:47.165456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:47.235458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1301/2626] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:51:47 2022] PUT /identity/v3/projects/5dde443064694864b3c43f234e5a2fc0/users/2c5005a536c746fe8c86bc03426e6062/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:47.246178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:47.246480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:47.246766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:47.274372 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] MFA Rules not processed for user `2c5005a536c746fe8c86bc03426e6062`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:47.275619 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:47.338560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1326/2627] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:47.349897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:47.351620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:47.428866 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dffbc-316f-44fc-98bf-1656d87fbf7a admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:47.434306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:47.434969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:47.435260 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:47.435540 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:47.444095 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:47.445678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40e0fa-9a9d-4b83-bef9-3c671d509876 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:47.454297 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1302/2628] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:57.194189 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:57.252874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f1fbb-35ea-4baf-956d-60feb40e905a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:57.257357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:57.258217 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:57.258496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:57.258768 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] PATH_INFO: `/v3/users/2c5005a536c746fe8c86bc03426e6062` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:57.259959 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5005a536c746fe8c86bc03426e6062)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:57.261851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:57.264150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:57.450430 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:57.538613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5005a536c746fe8c86bc03426e6062'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:57.539308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5005a536c746fe8c86bc03426e6062'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:57.546722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5005a536c746fe8c86bc03426e6062'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:57.646485 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5005a536c746fe8c86bc03426e6062'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:57.659832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5005a536c746fe8c86bc03426e6062'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:57.665782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1327/2629] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:51:57 2022] DELETE /identity/v3/users/2c5005a536c746fe8c86bc03426e6062 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:57.670727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:57.671078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:57.671361 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:57.688661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:57.689862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:57.747184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1303/2630] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:57.756728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:57.758154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:57.815709 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df29c-841f-4a7d-b1be-0dd9c52adb85 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:57.819335 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:57.819988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:57.820278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:57.820560 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:57.830199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:57.831659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562623f3-9563-4855-9c33-74404b846af9 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:57.841139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1328/2631] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:51:58.248378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:58.249893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:58.261786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d7b9-9ce2-4d26-8967-4c7c2ade3f0b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:58.266640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:58.267618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.268005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.268209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] PATH_INFO: `/v3/projects/5dde443064694864b3c43f234e5a2fc0` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.270169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dde443064694864b3c43f234e5a2fc0)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:58.272505 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:58.446225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dde443064694864b3c43f234e5a2fc0', 'request_id': 'req-28464d95-8003-4037-81fd-616061dd313e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:58.451871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dde443064694864b3c43f234e5a2fc0', 'request_id': 'req-28464d95-8003-4037-81fd-616061dd313e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:58.514897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dde443064694864b3c43f234e5a2fc0', 'request_id': 'req-28464d95-8003-4037-81fd-616061dd313e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:58.526310 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] The token cache is being invalidate because project 5dde443064694864b3c43f234e5a2fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:51:58.528198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:58.528198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-28464d95-8003-4037-81fd-616061dd313e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:51:58.528782 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1304/2632] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:51:58 2022] DELETE /identity/v3/projects/5dde443064694864b3c43f234e5a2fc0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:58.534144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.534651 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.534773 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.553044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:58.554291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:58.612391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1329/2633] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:58.619913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.620206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.620487 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.639137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:58.640331 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:58.648196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1305/2634] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:58.652170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:58.662204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b577d-656e-4ba2-837f-04f3b5e097cf None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:58.666807 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:58.667484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.667774 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.668059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.668784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:51:58.669099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:58.670693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:58.682035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1330/2635] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:51:58.686316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:58.695788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267072-b440-429a-ac63-91d7707fdfc5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:58.700409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:58.701543 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.701827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.702106 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.702807 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:58.704733 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:58.842750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1306/2636] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:51:58 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:51:58.846870 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:58.856930 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528e6b5-e170-4b51-a306-fe72a2724645 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:58.864801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:58.913351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:58.914024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:58.914454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:58.914780 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:58.915608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:58.917616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:58.993551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1331/2637] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:51:58 2022] POST /identity/v3/users => generated 303 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:51:58.997609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.008286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e89d7-3bb9-4042-9543-0296d7dbeb95 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.012911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.014150 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.014440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.014719 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.015490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.017082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.027348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1307/2638] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:51:59.031101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.041012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dd9f2-6b10-4c52-8639-5fb59f2d00b5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.045654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.046305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.046592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.046873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.047624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.049196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20bbc9-91a6-4b7e-9b36-e42c3a4731d3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.059010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1332/2639] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:51:59.063335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.073313 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db164cf-5b66-478d-9f3a-84badde34a9c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.078598 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.079524 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.079810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.080089 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] PATH_INFO: `/v3/projects/4bf1065f9b1241968389893ba3cca55c/users/bb3fec46cdf8465383e11d9530ad0b23/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.096265 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf1065f9b1241968389893ba3cca55c, user_id=bb3fec46cdf8465383e11d9530ad0b23, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.098841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.159993 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1308/2640] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:51:59 2022] PUT /identity/v3/projects/4bf1065f9b1241968389893ba3cca55c/users/bb3fec46cdf8465383e11d9530ad0b23/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:59.170416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.170772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.171084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.198628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] MFA Rules not processed for user `bb3fec46cdf8465383e11d9530ad0b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:51:59.199889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:51:59.267727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1333/2641] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:51:59.272307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.292683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:59.333245 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d290-2631-497f-ba26-452e9b5ab9a6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.338646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.342603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.342603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.342603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.342603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.342603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.442757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1309/2642] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:51:59 2022] POST /identity/v3/projects => generated 405 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:51:59.446853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.457678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc4067-7124-4b3d-b9d7-6742e2295585 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.519078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.519862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.520147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.520428 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.521118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.523060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.670616 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1334/2643] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:51:59 2022] POST /identity/v3/users => generated 309 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:51:59.674550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.684545 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6188101-1008-46b9-85d3-795fb83be770 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.689586 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.690485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.690770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.691094 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.691821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.693448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.703714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1310/2644] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:51:59.712330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.722897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692a15-52c6-4103-ae64-99ab43b2e483 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.727581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.728464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.728748 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.729027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] PATH_INFO: `/v3/projects/696526b651b34c8ebf14558255127162/users/b265ae79e46e49daa06894eb44752d5c/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.732689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:51:59.744569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696526b651b34c8ebf14558255127162, user_id=b265ae79e46e49daa06894eb44752d5c, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.747293 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:51:59.921665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1335/2645] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:51:59 2022] PUT /identity/v3/projects/696526b651b34c8ebf14558255127162/users/b265ae79e46e49daa06894eb44752d5c/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:51:59.925648 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:51:59.935491 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f21f-85c6-4aa3-b73a-b852fa3eff06 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:51:59.991529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:51:59.992195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:51:59.992485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:51:59.992768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:51:59.993501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:51:59.995126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde184d-46af-4cc6-a462-6f7fa8b2a1f5 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:00.004729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1311/2646] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:00.008806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:00.019476 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889d930-c715-40f1-bf5f-1b94245e80f5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:00.024011 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:00.024882 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.025164 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.025441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] PATH_INFO: `/v3/projects/696526b651b34c8ebf14558255127162/users/b265ae79e46e49daa06894eb44752d5c/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.027402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=696526b651b34c8ebf14558255127162, user_id=b265ae79e46e49daa06894eb44752d5c, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:00.030602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:00.040545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1336/2647] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:00 2022] PUT /identity/v3/projects/696526b651b34c8ebf14558255127162/users/b265ae79e46e49daa06894eb44752d5c/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:52:00.052373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.052741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.053033 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.080329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] MFA Rules not processed for user `b265ae79e46e49daa06894eb44752d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:00.081546 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:00.147851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1312/2648] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:00.159049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:00.160738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:00.217380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3822573-929e-470b-b154-d4cb453397ca admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:00.221108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:00.221754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.222037 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.222312 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.231349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:00.232955 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafae6de-6e88-41ec-a5ad-425fe5ac7eea service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:00.242256 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1337/2649] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:00 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:00.384831 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:00.402131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:00.446549 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd788fb-48ed-4ed2-9e3f-08693109c48f None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:00.451674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:00.452810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.453318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.454035 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] PATH_INFO: `/v3/users/bb3fec46cdf8465383e11d9530ad0b23` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.455372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3fec46cdf8465383e11d9530ad0b23)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:00.457426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:00.675754 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3fec46cdf8465383e11d9530ad0b23'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:00.676458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3fec46cdf8465383e11d9530ad0b23'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:00.683400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3fec46cdf8465383e11d9530ad0b23'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:00.787817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3fec46cdf8465383e11d9530ad0b23'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:00.800859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3fec46cdf8465383e11d9530ad0b23'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:00.806763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1313/2650] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:00 2022] DELETE /identity/v3/users/bb3fec46cdf8465383e11d9530ad0b23 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:00.811385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.811896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.812006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.830441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:00.831693 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:00.898184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1338/2651] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:00.908510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:00.909902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:00.966419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009246-2af8-4f94-b209-bb0ddc8a3afa admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:00.970526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:00.971198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:00.971484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:00.971759 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:00.980559 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:00.981969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:00.991318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1314/2652] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:01.464891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:01.466855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:01.479951 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357ac1f-eda5-4992-9954-9327addb51a9 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:01.484106 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:01.485012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:01.485295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:01.485569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] PATH_INFO: `/v3/projects/4bf1065f9b1241968389893ba3cca55c` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:01.486753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf1065f9b1241968389893ba3cca55c)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:01.488761 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:01.612402 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf1065f9b1241968389893ba3cca55c', 'request_id': 'req-10d8c09f-0789-4e09-a112-41fada97d18f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:01.618136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf1065f9b1241968389893ba3cca55c', 'request_id': 'req-10d8c09f-0789-4e09-a112-41fada97d18f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:01.754963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf1065f9b1241968389893ba3cca55c', 'request_id': 'req-10d8c09f-0789-4e09-a112-41fada97d18f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:01.762472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:01.766696 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] The token cache is being invalidate because project 4bf1065f9b1241968389893ba3cca55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:01.767013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:01.767634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:01.769160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1339/2653] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:01 2022] DELETE /identity/v3/projects/4bf1065f9b1241968389893ba3cca55c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:01.773257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:01.783008 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddea9df-4e4d-42fb-8688-6ba3f5d7e2ce service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:01.816673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:01.838579 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:01.839473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:01.839755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:01.840063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] PATH_INFO: `/v3/users/b265ae79e46e49daa06894eb44752d5c` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:01.841237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b265ae79e46e49daa06894eb44752d5c)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:01.843169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:02.099603 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b265ae79e46e49daa06894eb44752d5c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.100203 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b265ae79e46e49daa06894eb44752d5c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.106960 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b265ae79e46e49daa06894eb44752d5c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.175897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b265ae79e46e49daa06894eb44752d5c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.188397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-c72f191f-e409-4601-8327-dd413606ea1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b265ae79e46e49daa06894eb44752d5c'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.195259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1315/2654] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:01 2022] DELETE /identity/v3/users/b265ae79e46e49daa06894eb44752d5c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:02.653156 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:02.654668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:02.667654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c09f-0789-4e09-a112-41fada97d18f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:02.727249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:02.728118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:02.728407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:02.728686 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] PATH_INFO: `/v3/projects/696526b651b34c8ebf14558255127162` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:02.729859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696526b651b34c8ebf14558255127162)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:02.731908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:02.834638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696526b651b34c8ebf14558255127162', 'request_id': 'req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.840090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696526b651b34c8ebf14558255127162', 'request_id': 'req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.919060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696526b651b34c8ebf14558255127162', 'request_id': 'req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.935596 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] The token cache is being invalidate because project 696526b651b34c8ebf14558255127162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:02.935883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.936600 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-171a9611-d8f8-42b1-9b76-2d8a6ff2f922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:02.938131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1340/2655] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:02 2022] DELETE /identity/v3/projects/696526b651b34c8ebf14558255127162 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:02.943353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:02.943819 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:02.943942 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:02.961521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:02.962712 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:03.019869 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1316/2656] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:03.027425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.027717 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.027997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.046988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:03.048182 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:03.055944 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1341/2657] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:03.059861 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.069479 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25351a-20df-4058-b0e6-8783ab465a5b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.073974 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.074617 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.074904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.075214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.076472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:52:03.076780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.078376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.089661 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1317/2658] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:52:03.094292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.103324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d90b3-2e6b-4c1c-aa0b-10c802dbf24e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.107990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.108629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.108915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.109193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.109884 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.111809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.208064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1342/2659] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:03 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:03.212021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.222884 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add52867-8a94-4d86-91ca-afa0b16c8002 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.270832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:03.285604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.286264 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.286548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.286823 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.287539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.289449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.417370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1318/2660] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:03 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:03.421461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.431494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10389802-e41d-4127-9f46-9f259d4ed0b9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.436576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.437263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.437551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.437832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.438554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.440432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.450763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1343/2661] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:03.456226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.465997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56d79-f280-49b0-ba3d-b47b076788cc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.470553 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.471233 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.471522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.471802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.472527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.474134 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ba441-0d58-43f2-a5e6-82906b802661 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.483839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1319/2662] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:03.487722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.497314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992c9aa-9af1-4266-aee0-db1cf834c9fa admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.502280 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.503175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.503458 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.503737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] PATH_INFO: `/v3/projects/16594989c35b4dc2ab9a054141fc9d6d/users/4755599c391c4ef493e9108f85f940eb/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.519795 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=16594989c35b4dc2ab9a054141fc9d6d, user_id=4755599c391c4ef493e9108f85f940eb, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.524160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:03.622358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1344/2663] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:03 2022] PUT /identity/v3/projects/16594989c35b4dc2ab9a054141fc9d6d/users/4755599c391c4ef493e9108f85f940eb/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:03.632971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.633263 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.633547 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.665329 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] MFA Rules not processed for user `4755599c391c4ef493e9108f85f940eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:03.666602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:03.731762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1320/2664] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:03.736257 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:03.764590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:03.798059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddb26b-0cc8-45cb-a183-09697a129edc admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:03.802661 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:03.803380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:03.803639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:03.803916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:03.804594 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:03.806473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.007117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1345/2665] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:03 2022] POST /identity/v3/projects => generated 367 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:04.011292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.022220 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07661e88-0812-414b-818d-09ad1c5439cb None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.077185 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.077847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.078129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.078408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.079128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.081061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.218015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1321/2666] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:04 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:04.222250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.232323 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b76a4-8170-461e-ab6a-01bedd975a2b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.239002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.239002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.239002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.239002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.239387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.240494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.252345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1346/2667] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:04.256364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.266822 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8db621-ee54-4e9e-808b-3db50649ae83 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.271299 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.272171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.272458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.272736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] PATH_INFO: `/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b/users/b72fb48f2b26421ebca65ea214b7b168/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.288131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0eba98bdb7f4f3b8ec8fae016aabf3b, user_id=b72fb48f2b26421ebca65ea214b7b168, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.290775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.422378 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1322/2668] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:04 2022] PUT /identity/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b/users/b72fb48f2b26421ebca65ea214b7b168/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:04.426282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.436154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571ef3-cbe3-4889-a228-20bcc703062d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.490615 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.491310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.491598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.491878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.492598 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.494171 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e68059-79c1-468d-b6de-e08689fa166e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.503658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1347/2669] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:04.507664 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.517696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684908a-e436-488d-8b63-3b769fda12f0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.522100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.522998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.523285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.523567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] PATH_INFO: `/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b/users/b72fb48f2b26421ebca65ea214b7b168/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.525509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0eba98bdb7f4f3b8ec8fae016aabf3b, user_id=b72fb48f2b26421ebca65ea214b7b168, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.528149 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.538514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1323/2670] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:04 2022] PUT /identity/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b/users/b72fb48f2b26421ebca65ea214b7b168/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:04.550115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.550403 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.550684 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.581987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] MFA Rules not processed for user `b72fb48f2b26421ebca65ea214b7b168`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:04.583247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:04.651218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1348/2671] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:04.662368 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:04.664070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:04.670511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:04.721341 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512bb6b-e9a0-4212-9ef8-403df9c25a1f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:04.725040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:04.744699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:04.744699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:04.744699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:04.744699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:04.744699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba7bd7-dcb2-41a1-b84c-535990f686ec service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:04.745718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1324/2672] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:08.223495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:08.243897 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:08.285357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40637764-eb66-43b5-bbbc-15cf01146c6d None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:08.289874 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:08.290741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:08.291059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:08.291344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] PATH_INFO: `/v3/users/4755599c391c4ef493e9108f85f940eb` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:08.292641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4755599c391c4ef493e9108f85f940eb)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:08.294547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:08.490117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4755599c391c4ef493e9108f85f940eb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:08.490778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4755599c391c4ef493e9108f85f940eb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:08.498125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4755599c391c4ef493e9108f85f940eb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:08.617013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4755599c391c4ef493e9108f85f940eb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:08.629754 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4755599c391c4ef493e9108f85f940eb'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:08.635710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1349/2673] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:08 2022] DELETE /identity/v3/users/4755599c391c4ef493e9108f85f940eb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:08.640718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:08.641194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:08.641675 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:08.658461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:08.660238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:08.721189 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1325/2674] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:08.731199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:08.732914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:08.792705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1d0b0-cc80-4fe9-ad93-52b8c2a0c157 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:08.797158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:08.797857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:08.798167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:08.798444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:08.807808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:08.809218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:08.818859 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1350/2675] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:09.428637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:09.431858 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:09.442839 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9be57-f88d-482f-9531-0d35a7b8bc61 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:09.447442 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:09.448298 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:09.448582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:09.448860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] PATH_INFO: `/v3/projects/16594989c35b4dc2ab9a054141fc9d6d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:09.450054 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16594989c35b4dc2ab9a054141fc9d6d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:09.452031 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:09.665145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16594989c35b4dc2ab9a054141fc9d6d', 'request_id': 'req-45aec26e-28a2-43e4-8cc2-1eea16d7f707'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:09.671367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16594989c35b4dc2ab9a054141fc9d6d', 'request_id': 'req-45aec26e-28a2-43e4-8cc2-1eea16d7f707'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:09.822649 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16594989c35b4dc2ab9a054141fc9d6d', 'request_id': 'req-45aec26e-28a2-43e4-8cc2-1eea16d7f707'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:09.839409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] The token cache is being invalidate because project 16594989c35b4dc2ab9a054141fc9d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:09.839615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:09.840285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:09.842373 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1326/2676] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:09 2022] DELETE /identity/v3/projects/16594989c35b4dc2ab9a054141fc9d6d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:09.846863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:09.861090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6890-618c-480f-8f94-0a0175970ba9 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:09.911373 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:09.928847 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:09.929728 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:09.930107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:09.930388 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] PATH_INFO: `/v3/users/b72fb48f2b26421ebca65ea214b7b168` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:09.931634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72fb48f2b26421ebca65ea214b7b168)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:09.933642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:10.230945 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72fb48f2b26421ebca65ea214b7b168'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:10.233720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72fb48f2b26421ebca65ea214b7b168'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:10.243120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72fb48f2b26421ebca65ea214b7b168'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:10.309205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72fb48f2b26421ebca65ea214b7b168'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:10.325597 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-0a1a4db2-9720-4c84-8a88-b4891f8a17e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72fb48f2b26421ebca65ea214b7b168'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:10.332542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1351/2677] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:09 2022] DELETE /identity/v3/users/b72fb48f2b26421ebca65ea214b7b168 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:10.803034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:10.803583 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:10.820577 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec26e-28a2-43e4-8cc2-1eea16d7f707 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:10.857488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:10.885676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:10.886544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:10.886826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:10.887126 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] PATH_INFO: `/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:10.888511 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0eba98bdb7f4f3b8ec8fae016aabf3b)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:10.890449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.080182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0eba98bdb7f4f3b8ec8fae016aabf3b', 'request_id': 'req-d516a277-8929-47e5-9f0d-24fc1366a560'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:11.085574 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0eba98bdb7f4f3b8ec8fae016aabf3b', 'request_id': 'req-d516a277-8929-47e5-9f0d-24fc1366a560'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:11.206288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0eba98bdb7f4f3b8ec8fae016aabf3b', 'request_id': 'req-d516a277-8929-47e5-9f0d-24fc1366a560'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:11.219779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] The token cache is being invalidate because project a0eba98bdb7f4f3b8ec8fae016aabf3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:11.220138 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:11.220765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d516a277-8929-47e5-9f0d-24fc1366a560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:11.222279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1327/2678] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:10 2022] DELETE /identity/v3/projects/a0eba98bdb7f4f3b8ec8fae016aabf3b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:11.227119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.227613 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.227731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.245809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:11.247174 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:11.308934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1352/2679] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:11.316871 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.317168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.317443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.335856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:11.337048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:11.345325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1328/2680] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:11.349357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.359453 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e17beb-927b-4641-b346-299d19210bfb None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.364104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.364763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.365077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.365998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.366154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:52:11.366421 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.368184 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.379800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1353/2681] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:52:11.384055 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.393883 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6a690-43a4-4fa3-8c27-540953178e13 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.398492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.399364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.406070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.406070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.406070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.406070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.503783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1329/2682] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:11 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:11.507938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.518375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e29e0-98e9-489e-abb1-9fb3a77e3ff2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.562977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:11.574493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.577364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.577364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.577364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.577364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.578460 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.679853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1354/2683] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:11 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:11.683943 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.694818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d231-b0ec-4c6e-92a6-01e2010752fc admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.699461 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.700112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.700396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.700678 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.701405 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.703868 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.717370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1330/2684] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:11.721477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.732131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c363dd-1c7f-423a-9dac-8e4f19740c76 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.736962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.737617 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.737903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.738185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.738907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.740521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fab69-fdcd-443f-bbce-d0eafbdf7050 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.751364 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1355/2685] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:11.755578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:11.765321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d43c26-20e9-4e87-8c67-a38cd85080e6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:11.769859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:11.770723 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.771040 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.771317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] PATH_INFO: `/v3/projects/483d7ae24f604a91923f28c59d76796a/users/4d3043b6b5284db8b2590cde036cc583/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.786937 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483d7ae24f604a91923f28c59d76796a, user_id=4d3043b6b5284db8b2590cde036cc583, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:11.789585 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:11.914913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1331/2686] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:11 2022] PUT /identity/v3/projects/483d7ae24f604a91923f28c59d76796a/users/4d3043b6b5284db8b2590cde036cc583/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:11.926647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:11.926957 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:11.927247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:11.954316 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] MFA Rules not processed for user `4d3043b6b5284db8b2590cde036cc583`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:11.955552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:12.018724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1356/2687] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:12.023216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.081793 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:12.085637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3082-0872-4873-882b-78b502fa1314 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.090365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.091169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.091456 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.094318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.094318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.094632 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.179862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1332/2688] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:12 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:12.183406 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.194990 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01736946-70d8-4c74-9d68-08d6d69aff05 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.250699 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.251427 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.251711 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.251987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.252694 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.254627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.358561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1357/2689] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:12 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:12.361305 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.371988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a8ebd-df55-47d7-bbf2-9afc689666db admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.377308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.378048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.378693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.378693 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.380136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.381914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.393081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1333/2690] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:12.398167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.409790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feecd23-3e06-491d-b860-381c0c52e423 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.415223 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.415505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.415791 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.416062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] PATH_INFO: `/v3/projects/fe8223d812114f85acdff0e7f5ca7945/users/bada5487f67e41e7aad6a7a079418c2d/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.432113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8223d812114f85acdff0e7f5ca7945, user_id=bada5487f67e41e7aad6a7a079418c2d, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.435305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.567056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1358/2691] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:12 2022] PUT /identity/v3/projects/fe8223d812114f85acdff0e7f5ca7945/users/bada5487f67e41e7aad6a7a079418c2d/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:52:12.571270 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.581328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62ed0-5470-450d-ad88-ba4a09da9ac6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.654507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.655303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.655655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.656014 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.656896 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.658548 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453aac8-abdf-4821-818b-e01bdc98d337 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.670817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1334/2692] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:12.674689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.687500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17166752-d38b-49b1-a9ab-61aa9e9a03e1 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.693258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.694371 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.694757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.695305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] PATH_INFO: `/v3/projects/fe8223d812114f85acdff0e7f5ca7945/users/bada5487f67e41e7aad6a7a079418c2d/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.697344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8223d812114f85acdff0e7f5ca7945, user_id=bada5487f67e41e7aad6a7a079418c2d, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.700016 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.713654 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1359/2693] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:12 2022] PUT /identity/v3/projects/fe8223d812114f85acdff0e7f5ca7945/users/bada5487f67e41e7aad6a7a079418c2d/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:12.724342 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.724639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.724945 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.758792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] MFA Rules not processed for user `bada5487f67e41e7aad6a7a079418c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:12.760123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:12.830198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1335/2694] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:12.842034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:12.843926 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:12.854805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:12.900557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b004f-f143-4173-b5c3-087a6fd5a5c3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:12.904752 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:12.905420 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:12.905730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:12.906007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:12.914857 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:12.916287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:12.925770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1360/2695] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:12 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:13.067060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:13.077237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644803d4-6bba-4350-a411-97278c30ad4b None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:13.080954 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:13.081536 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:13.081821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:13.082121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:13.129289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:13.138238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:13.139736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:13.148499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1336/2696] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:16.068767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:16.070185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:16.128580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915202-7e3f-40af-8f26-769bb8b86e18 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:16.132517 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:16.138416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:16.138416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:16.138416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:16.138416 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:16.200052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:16.201484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46091cad-485c-4811-9016-71c91bb92765 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:16.211206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1361/2697] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:18.506653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:18.517919 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831d975-ea2f-41ed-9296-73586c64c6c6 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:18.522668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:18.523618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:18.523902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:18.524180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] PATH_INFO: `/v3/users/4d3043b6b5284db8b2590cde036cc583` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:18.525365 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3043b6b5284db8b2590cde036cc583)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:18.527332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:18.839061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3043b6b5284db8b2590cde036cc583'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:18.839743 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3043b6b5284db8b2590cde036cc583'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:18.846848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3043b6b5284db8b2590cde036cc583'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:18.947145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3043b6b5284db8b2590cde036cc583'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:18.959730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3043b6b5284db8b2590cde036cc583'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:18.966279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1337/2698] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:18 2022] DELETE /identity/v3/users/4d3043b6b5284db8b2590cde036cc583 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:18.970732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:18.971272 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:18.971391 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:18.989188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:18.990396 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:19.051347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1362/2699] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:19.065432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:19.066781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:19.123594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4ae86-cddb-427f-b6f5-29dfd3149cb8 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:19.128932 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:19.129597 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:19.129894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:19.130177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:19.141901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:19.143409 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:19.162889 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1338/2700] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:19.773079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:19.774514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:19.781444 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:19.786845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a0e37-4b95-4cfe-8669-aad4d5495e96 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:19.791997 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:19.792947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:19.793241 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:19.794678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] PATH_INFO: `/v3/projects/483d7ae24f604a91923f28c59d76796a` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:19.796937 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483d7ae24f604a91923f28c59d76796a)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:19.800187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:19.932368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483d7ae24f604a91923f28c59d76796a', 'request_id': 'req-93225b93-a7e2-457b-ae6f-f64b96d5a543'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:19.938326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483d7ae24f604a91923f28c59d76796a', 'request_id': 'req-93225b93-a7e2-457b-ae6f-f64b96d5a543'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.040004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483d7ae24f604a91923f28c59d76796a', 'request_id': 'req-93225b93-a7e2-457b-ae6f-f64b96d5a543'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.053556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] The token cache is being invalidate because project 483d7ae24f604a91923f28c59d76796a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:20.053852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.054507 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.056115 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1363/2701] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:19 2022] DELETE /identity/v3/projects/483d7ae24f604a91923f28c59d76796a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:20.060160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:20.070628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67721e9-3e04-427d-b5ba-4140dd240e40 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:20.128800 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:20.129697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:20.129985 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:20.130266 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] PATH_INFO: `/v3/users/bada5487f67e41e7aad6a7a079418c2d` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:20.131483 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bada5487f67e41e7aad6a7a079418c2d)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:20.133484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:20.138734 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:20.169467 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:20.170998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:20.188376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93225b93-a7e2-457b-ae6f-f64b96d5a543 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:20.193326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:20.194250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:20.194538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:20.195631 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] PATH_INFO: `/v3/users/8b5a188cee0d4727a14153088b8d95c6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:20.196148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5a188cee0d4727a14153088b8d95c6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:20.198060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:20.487509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bada5487f67e41e7aad6a7a079418c2d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.488175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bada5487f67e41e7aad6a7a079418c2d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.495061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bada5487f67e41e7aad6a7a079418c2d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.579480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5a188cee0d4727a14153088b8d95c6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.580126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5a188cee0d4727a14153088b8d95c6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.587038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5a188cee0d4727a14153088b8d95c6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.599711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bada5487f67e41e7aad6a7a079418c2d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.612701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bada5487f67e41e7aad6a7a079418c2d'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.618485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1339/2702] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:20 2022] DELETE /identity/v3/users/bada5487f67e41e7aad6a7a079418c2d => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:52:20.656879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5a188cee0d4727a14153088b8d95c6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.671000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5a188cee0d4727a14153088b8d95c6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:20.677295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1364/2703] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:20 2022] DELETE /identity/v3/users/8b5a188cee0d4727a14153088b8d95c6 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:20.982387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:20.984007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.035865 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.039797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.055904 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5bb50-a26c-4976-a6b6-dc5593bf7817 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.108627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:21.114006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dddfe-1d35-4bf3-9468-f56f5494e51c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.119764 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.120669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.120962 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.121245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] PATH_INFO: `/v3/projects/fe8223d812114f85acdff0e7f5ca7945` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.122550 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8223d812114f85acdff0e7f5ca7945)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.124484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.125107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.125973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.126261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.126542 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] PATH_INFO: `/v3/projects/38629fbda01748eb98e2dca20a3da10a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.127792 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38629fbda01748eb98e2dca20a3da10a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.129697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.241031 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8223d812114f85acdff0e7f5ca7945', 'request_id': 'req-63a671bd-9eaa-434d-a80e-c01abdc7e66c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.246565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8223d812114f85acdff0e7f5ca7945', 'request_id': 'req-63a671bd-9eaa-434d-a80e-c01abdc7e66c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.248291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:21.257402 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38629fbda01748eb98e2dca20a3da10a', 'request_id': 'req-e43ee62c-87c5-4311-9895-05ce01c2b4d1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.263272 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38629fbda01748eb98e2dca20a3da10a', 'request_id': 'req-e43ee62c-87c5-4311-9895-05ce01c2b4d1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.377077 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8223d812114f85acdff0e7f5ca7945', 'request_id': 'req-63a671bd-9eaa-434d-a80e-c01abdc7e66c'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.388491 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] The token cache is being invalidate because project fe8223d812114f85acdff0e7f5ca7945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:21.388782 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.389408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-63a671bd-9eaa-434d-a80e-c01abdc7e66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.390980 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1365/2704] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:21 2022] DELETE /identity/v3/projects/fe8223d812114f85acdff0e7f5ca7945 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:52:21.395757 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbda2-99d9-4592-85fe-4a7331e03836 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.396051 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbda2-99d9-4592-85fe-4a7331e03836 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.396336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbda2-99d9-4592-85fe-4a7331e03836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.413892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-516cbda2-99d9-4592-85fe-4a7331e03836 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:21.415128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-516cbda2-99d9-4592-85fe-4a7331e03836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.445718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38629fbda01748eb98e2dca20a3da10a', 'request_id': 'req-e43ee62c-87c5-4311-9895-05ce01c2b4d1'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.457355 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] The token cache is being invalidate because project 38629fbda01748eb98e2dca20a3da10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:21.458010 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.459050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-e43ee62c-87c5-4311-9895-05ce01c2b4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:21.467381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1340/2705] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:20 2022] DELETE /identity/v3/projects/38629fbda01748eb98e2dca20a3da10a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:21.467381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d691-a8cc-4ef4-ab04-5c39f5cb7358 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.467381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d691-a8cc-4ef4-ab04-5c39f5cb7358 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.467381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d691-a8cc-4ef4-ab04-5c39f5cb7358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.474366 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1366/2706] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:21.482542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.482833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.483140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.484922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fcc8d691-a8cc-4ef4-ab04-5c39f5cb7358 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:21.486153 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fcc8d691-a8cc-4ef4-ab04-5c39f5cb7358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.494172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1341/2707] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:21.500642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:21.502510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.502633 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.502753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.502753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.510245 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1367/2708] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:21.513983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.519801 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:21.520992 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:21.522753 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13d03-2db6-4a80-8cb4-d3de851227ab None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.528766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.536296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1342/2709] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.536419 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.539000 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.548004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1368/2710] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:52:21.548240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf194-90a9-4aca-8513-160fc286d7df None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.548949 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.552846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.553494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.553777 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.554050 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.554770 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:52:21.555113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.556697 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.559918 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d3e9-c8af-4684-b988-6821a365f7fb admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.564871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.565524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.565811 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.566091 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.566775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.568700 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.569196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1343/2711] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:52:21.574129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.584100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed6869-447c-47e5-af81-acc6592aa7ab admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.588696 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.589330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.589613 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.589886 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.590592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.592501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.715676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1369/2712] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/projects => generated 409 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:21.719705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.731911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1344/2713] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:21.737026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.770953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:21.784965 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c85bc-8f3c-45fa-b296-efc9706c8def admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.797127 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4ca35-522a-41d0-a2e0-9d96592f45af admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.801826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.802492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.802780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.803174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.803867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.805790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.843185 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.843862 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.844146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.844423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.845108 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.847043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.946890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1345/2714] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:21.950997 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.960255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71948eb-0375-437f-a2f3-8ede92ef1717 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.965171 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.965816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.966107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.966391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.967158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.968749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:21.983690 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1370/2715] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:21 2022] POST /identity/v3/users => generated 305 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:21.983901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1346/2716] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:52:21.984017 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.984717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:21.991883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6816727-07f2-4cd2-abf6-b847cf39c00e admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.994516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d0f2-08ed-4c65-91dd-006de5d4dd3e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:21.997279 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:21.997939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:21.998227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:21.998509 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:21.999275 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:21.999780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.000518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.000837 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.000964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.001137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.001892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.003527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.010841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1371/2717] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:52:22.016422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1347/2718] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:52:22.016590 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.017388 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.024502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b5bf-0c05-49fe-a84a-e6ca80fc394c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.026900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a815a4-9d75-443a-ba41-6daa1b4d24d6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.029098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.029745 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.030035 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.030394 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.031078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.035640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.035640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.035863 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.035988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.035988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] PATH_INFO: `/v3/projects/7caa2a2f7c644b16a76b9dff92a2b2e8/users/0251939fa06a45da898a12a37ac91163/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.043242 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1372/2719] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:52:22.047917 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.053781 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7caa2a2f7c644b16a76b9dff92a2b2e8, user_id=0251939fa06a45da898a12a37ac91163, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.056645 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bae5f-b212-4384-b7e8-ef5d11e79e00 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.057249 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2c566-8d76-42c6-b213-32851cb4e3af admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.063433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.064346 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.064907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.065216 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] PATH_INFO: `/v3/projects/2586b0aa398449ef8836462f6c585ca2/users/7f6d00d864264c7ca5b3b30387bd20cf/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.079629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2586b0aa398449ef8836462f6c585ca2, user_id=7f6d00d864264c7ca5b3b30387bd20cf, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.082243 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348963a-104d-4d5f-925b-ed82f1707fd4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.166289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1348/2720] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:22 2022] PUT /identity/v3/projects/7caa2a2f7c644b16a76b9dff92a2b2e8/users/0251939fa06a45da898a12a37ac91163/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:22.171766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.172110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.172396 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.191030 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:22.192239 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:22.200479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1373/2721] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:22 2022] PUT /identity/v3/projects/2586b0aa398449ef8836462f6c585ca2/users/7f6d00d864264c7ca5b3b30387bd20cf/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:22.204358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:22.211640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.212079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.212461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.240023 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] MFA Rules not processed for user `7f6d00d864264c7ca5b3b30387bd20cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:22.241296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:22.251352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1349/2722] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:22.261739 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.265104 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:22.306889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1374/2723] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:52:22.311187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.320041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508fbc9-4117-45f6-9b64-6d151600fd85 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.321638 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795ae6e-4fba-453b-b64d-ec63b8fbf8c8 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.323698 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.324346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.324635 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.325001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.326479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.327151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.327439 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.327718 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.328400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.330341 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.334417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.335888 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.346053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1350/2724] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:52:22.441552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1375/2725] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:22 2022] POST /identity/v3/projects => generated 409 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:22.445595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.455362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7dab-2eda-490c-a448-e1c1eee58533 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.514432 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.515170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.515458 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.515740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.516640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.518562 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.599973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1351/2726] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:22 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:52:22.603867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.610747 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:22.614541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398951da-1259-4b2a-bc02-c6474630ce8d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.619637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.623839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.623839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.623839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.623839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.623839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.634739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1376/2727] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:22.639360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.649714 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538678-3fc5-408e-af4b-6b0e7251ca54 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.655229 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.664468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.664468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.664468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] PATH_INFO: `/v3/projects/86d7e3bb606e421b8a99130f9545d0aa/users/d125f109aabd4b90b6536931213a7902/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.672345 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7e3bb606e421b8a99130f9545d0aa, user_id=d125f109aabd4b90b6536931213a7902, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.674976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.749045 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1352/2728] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:22 2022] PUT /identity/v3/projects/86d7e3bb606e421b8a99130f9545d0aa/users/d125f109aabd4b90b6536931213a7902/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:22.753177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.764116 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1549b94-0c96-449e-adc7-e47b4fd26141 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.821832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.822503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.822789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.823101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.823827 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.825927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dd0ea-b648-4a93-a90e-1134f947e71e admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.835545 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1377/2729] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:22.839700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.850478 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8b270-798b-4265-ad81-4f75da5a3d0f admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:22.855021 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:22.855913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.856198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.856481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] PATH_INFO: `/v3/projects/86d7e3bb606e421b8a99130f9545d0aa/users/d125f109aabd4b90b6536931213a7902/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.858434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7e3bb606e421b8a99130f9545d0aa, user_id=d125f109aabd4b90b6536931213a7902, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:22.861078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:22.871805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1353/2730] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:22 2022] PUT /identity/v3/projects/86d7e3bb606e421b8a99130f9545d0aa/users/d125f109aabd4b90b6536931213a7902/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:22.882160 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:22.882479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:22.882755 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:22.909766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] MFA Rules not processed for user `d125f109aabd4b90b6536931213a7902`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:22.911014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:22.983749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1378/2731] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:22.994475 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:22.996346 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:23.015285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:23.063920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc8d9-3fe2-4b03-b884-cca1422511bb admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:23.068028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:23.068674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:23.068959 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:23.069237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:23.078991 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:23.080436 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e01d3-bd23-442c-9a1f-b0f1b1ae3316 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:23.090036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1354/2732] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:23.457852 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:23.467568 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f10db8-5943-4b62-9e7a-43ab0a370702 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:23.471214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:23.471858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:23.472146 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:23.472426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:23.529229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:23.530673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:23.539876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1379/2733] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:27.933852 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95953-c4e2-486d-8eef-6cb6096cda88 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:27.934516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95953-c4e2-486d-8eef-6cb6096cda88 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:27.934516 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95953-c4e2-486d-8eef-6cb6096cda88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:27.963288 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c8d95953-c4e2-486d-8eef-6cb6096cda88 None None] MFA Rules not processed for user `0251939fa06a45da898a12a37ac91163`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:27.964571 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c8d95953-c4e2-486d-8eef-6cb6096cda88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:28.031258 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1355/2734] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:28.047330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:28.048737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:28.088488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=100 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:28.158784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f43fc-b1c8-44c4-b827-8fb84c2180fe service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:28.170117 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:28.170821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:28.171181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:28.171495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:28.183469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:28.185671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:28.195939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1380/2735] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:31.032397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:31.033294 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:31.033821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:31.056300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] MFA Rules not processed for user `0251939fa06a45da898a12a37ac91163`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:31.058142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:31.067718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1356/2736] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:31.424464 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:31.426000 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:31.490778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d5bc-d74f-4cde-b852-473f7d040738 service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:31.494463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:31.495161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:31.495452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:31.495738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:31.498385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:31.509313 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:31.510751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674027f1-a7b0-423b-92b6-741fa0d892d3 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:31.519667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1381/2737] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:34.234023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:34.294885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152ee72-33e0-4fab-a422-5cb54f9d886a None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:34.299406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:34.300267 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:34.300555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:34.300836 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] PATH_INFO: `/v3/users/7f6d00d864264c7ca5b3b30387bd20cf` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:34.302023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6d00d864264c7ca5b3b30387bd20cf)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:34.304004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:34.551464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:34.556651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6d00d864264c7ca5b3b30387bd20cf'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:34.557328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6d00d864264c7ca5b3b30387bd20cf'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:34.565348 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6d00d864264c7ca5b3b30387bd20cf'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:34.643950 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6d00d864264c7ca5b3b30387bd20cf'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:34.656620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6d00d864264c7ca5b3b30387bd20cf'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:34.662618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1357/2738] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:34 2022] DELETE /identity/v3/users/7f6d00d864264c7ca5b3b30387bd20cf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:34.667302 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:34.667797 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:34.668408 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:34.686361 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:34.687620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:34.760544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1382/2739] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:34.769717 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:34.771292 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:34.836623 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c59c4b-3f5a-4535-84e3-6e7ed103e4c3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:34.845137 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:34.845779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:34.846060 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:34.846333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:34.856114 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:34.857711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:34.868741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1358/2740] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:35.384548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:35.386167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:35.406879 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ddae9-1f35-4972-b18f-2b1b51d3fa41 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:35.413499 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:35.414349 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:35.414647 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:35.414983 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] PATH_INFO: `/v3/projects/2586b0aa398449ef8836462f6c585ca2` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:35.416157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2586b0aa398449ef8836462f6c585ca2)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:35.418830 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:35.555090 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:35.565547 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2586b0aa398449ef8836462f6c585ca2', 'request_id': 'req-b168ef18-3fad-46f2-9733-7374a8958f2d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:35.572572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2586b0aa398449ef8836462f6c585ca2', 'request_id': 'req-b168ef18-3fad-46f2-9733-7374a8958f2d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:35.719525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2586b0aa398449ef8836462f6c585ca2', 'request_id': 'req-b168ef18-3fad-46f2-9733-7374a8958f2d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:35.731938 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] The token cache is being invalidate because project 2586b0aa398449ef8836462f6c585ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:35.732229 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:35.736147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:35.736147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1383/2741] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:35 2022] DELETE /identity/v3/projects/2586b0aa398449ef8836462f6c585ca2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:35.738640 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:35.748594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a70f-1564-4399-add4-7b9f5580aee4 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:35.806976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:35.807590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:35.807878 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:35.808158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] PATH_INFO: `/v3/users/d125f109aabd4b90b6536931213a7902` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:35.809393 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d125f109aabd4b90b6536931213a7902)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:35.811358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:36.206467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd125f109aabd4b90b6536931213a7902'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:36.207390 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd125f109aabd4b90b6536931213a7902'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:36.224876 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd125f109aabd4b90b6536931213a7902'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:36.359318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd125f109aabd4b90b6536931213a7902'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:36.361522 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:36.373356 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-9f8beb98-1419-4ea5-9329-965bb14b85ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd125f109aabd4b90b6536931213a7902'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:36.379503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1359/2742] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:35 2022] DELETE /identity/v3/users/d125f109aabd4b90b6536931213a7902 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:36.908418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:36.909888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:36.927602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168ef18-3fad-46f2-9733-7374a8958f2d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:36.984536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:36.985074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:36.985449 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:36.985637 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] PATH_INFO: `/v3/projects/86d7e3bb606e421b8a99130f9545d0aa` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:36.986914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d7e3bb606e421b8a99130f9545d0aa)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:36.988845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:37.243737 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d7e3bb606e421b8a99130f9545d0aa', 'request_id': 'req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:37.249287 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d7e3bb606e421b8a99130f9545d0aa', 'request_id': 'req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:37.382002 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d7e3bb606e421b8a99130f9545d0aa', 'request_id': 'req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:37.393315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] The token cache is being invalidate because project 86d7e3bb606e421b8a99130f9545d0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:37.393604 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:37.394226 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-61e9bd1d-4f4a-43fd-8a58-53356fa9504d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:37.395851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1384/2743] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:36 2022] DELETE /identity/v3/projects/86d7e3bb606e421b8a99130f9545d0aa => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:37.400691 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.401338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.401338 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.421702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:37.421702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:37.479349 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1360/2744] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:37.487652 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.487946 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.488227 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.505935 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:37.507180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:37.514850 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1385/2745] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:37.518856 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:37.528916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cea823-0c97-42c3-a7ba-168a1faf802e None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:37.533499 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:37.534144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.534433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.534713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.535496 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:52:37.535804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:37.537398 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:37.549340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1361/2746] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:52:37.553263 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:37.562570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582aec2-47bb-433a-b693-cd033672ad65 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:37.567495 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:37.568148 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.568436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.568719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.569407 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:37.571370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:37.708337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1386/2747] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:52:37 2022] POST /identity/v3/projects => generated 359 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:37.714145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:37.725367 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d9f5e-769a-4b2d-b293-a9ae8af6d8c5 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:37.782450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:37.783158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.783446 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.783727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.784460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:37.786370 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:37.925005 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1362/2748] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:52:37 2022] POST /identity/v3/users => generated 280 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:52:37.937721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:37.948606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0fb53-6be3-4a25-9e92-5fc109b73772 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:37.953839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:37.954443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.954730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.955043 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:37.955771 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:37.957368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:37.969730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1387/2749] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:52:37.983728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:37.993604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f64d-73f9-4d14-9954-cca80854f35e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:37.998636 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:37.999314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:37.999600 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:37.999875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.000612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:38.002175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ba705-6c3b-4f4f-98bc-baf2c0c1a832 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:38.012699 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1363/2750] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:52:38.016832 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:38.026900 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e014d-1f96-47ff-b04b-656a442a12e3 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:38.031496 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:38.037841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:38.037841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:38.037841 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] PATH_INFO: `/v3/projects/aef6d7b63d9a41639d512586d0c17397/users/3a05cd07dabd4b91a0fe8bc25e063fbb/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.046069 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:38.049099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef6d7b63d9a41639d512586d0c17397, user_id=3a05cd07dabd4b91a0fe8bc25e063fbb, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:38.051794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:38.221170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1388/2751] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:52:38 2022] PUT /identity/v3/projects/aef6d7b63d9a41639d512586d0c17397/users/3a05cd07dabd4b91a0fe8bc25e063fbb/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:38.223022 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:38.223316 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:38.223594 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.250618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] MFA Rules not processed for user `3a05cd07dabd4b91a0fe8bc25e063fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:38.251893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:38.285935 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:38.318034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1364/2752] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:38.350607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:38.352310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:38.409188 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e026-54cb-4fdc-8aae-9837564f776b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:38.412947 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:38.413601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:38.413888 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:38.414170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.425540 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:38.429340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034c3db-14bf-46d6-9c5c-58ad021464b6 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:38.438503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1389/2753] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:38 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:38.454397 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:38.519644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94514396-255a-4c77-ac44-0589a408ea85 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:38.524677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:38.525629 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:38.525916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:38.526318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] PATH_INFO: `/v3/users/3a05cd07dabd4b91a0fe8bc25e063fbb` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.528016 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a05cd07dabd4b91a0fe8bc25e063fbb)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:38.530028 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:38.805655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a05cd07dabd4b91a0fe8bc25e063fbb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:38.805655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a05cd07dabd4b91a0fe8bc25e063fbb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:38.812752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a05cd07dabd4b91a0fe8bc25e063fbb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:38.957291 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a05cd07dabd4b91a0fe8bc25e063fbb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:38.969818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a05cd07dabd4b91a0fe8bc25e063fbb'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:38.975768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1365/2754] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:52:38 2022] DELETE /identity/v3/users/3a05cd07dabd4b91a0fe8bc25e063fbb => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:38.980218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:38.980703 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:38.980853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:38.997876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:52:38.999095 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:39.055822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1390/2755] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:52:39.066217 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:39.067669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:39.125518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b13b-e453-429f-ad36-da7b111ef0f9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:39.129081 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:39.129730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:39.130013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:39.130293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:39.139268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:39.140692 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:39.144529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:39.150486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1366/2756] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:52:39.749317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:52:39.749317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:52:39.749317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028df2a-8a70-43c3-ac3a-e5c2c3bf6261 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:52:39.750042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:52:39.750042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:52:39.750042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:52:39.750042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] PATH_INFO: `/v3/projects/aef6d7b63d9a41639d512586d0c17397` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:52:39.750915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef6d7b63d9a41639d512586d0c17397)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:52:39.750915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:52:39.877854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef6d7b63d9a41639d512586d0c17397', 'request_id': 'req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:39.883525 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef6d7b63d9a41639d512586d0c17397', 'request_id': 'req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:40.117478 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef6d7b63d9a41639d512586d0c17397', 'request_id': 'req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:40.129986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] The token cache is being invalidate because project aef6d7b63d9a41639d512586d0c17397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:52:40.130276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:40.130916 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:52:40.132524 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1391/2757] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:52:39 2022] DELETE /identity/v3/projects/aef6d7b63d9a41639d512586d0c17397 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:52:49.859612 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:52:50.127340 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:53:35.605350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:53:35.606813 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:53:35.618715 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1b10-4daf-4302-98e8-84fa14852967 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:53:35.668668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:53:35.669315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:53:35.669599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:53:35.669875 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:53:35.671694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:53:35.742558 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:53:35.744039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:53:35.755595 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1367/2758] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:53:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:53:36.924856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:53:36.926336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:53:36.938821 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdf2f-1db5-428a-b55e-4d561cf7bae2 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:53:36.945140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:53:36.945140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:53:36.945140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:53:36.945140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:53:36.946042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:53:36.960065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:53:36.961539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:53:36.972510 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1392/2759] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:53:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:53:45.750066 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=54 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:53:46.967605 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:28.316659 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:28.318938 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:28.336780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3df92e-86af-4ec7-a7ad-a777a2952733 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:28.345158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:28.345846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:28.346115 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:28.346423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:28.348317 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:28.362273 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:28.363914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:28.375578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1368/2760] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:54:28.450581 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:28.461018 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c530-b65b-4bf2-9345-825cbb7842dd service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:28.463576 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:28.464224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:28.464554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:28.465383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:28.467317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:28.477441 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:28.478876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:28.487928 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1393/2761] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:54:31.071046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:31.072557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:31.086450 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff54d-ae55-4d1f-a401-afb9c2471798 service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:31.092468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:31.093441 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:31.093729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:31.094007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:31.095990 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:31.109147 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:31.110753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:31.122498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1369/2762] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:54:35.848498 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:35.850075 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:35.862318 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e09d-210a-47e1-b3ad-e28ac2d1b7fc service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:35.869533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:35.869533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:35.869533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:35.869533 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:35.869963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:35.931772 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:35.933195 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:35.943585 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1394/2763] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:54:40.760922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:40.762544 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:40.776380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5864-f1ad-4a59-a05e-95cb4919ed1d service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:40.781387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:40.782472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:40.782786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:40.783113 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] PATH_INFO: `/v3/users/ab96fb75e716433f857cb9a4c0d7fb2a` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:40.784279 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab96fb75e716433f857cb9a4c0d7fb2a)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:40.786318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:40.954183 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab96fb75e716433f857cb9a4c0d7fb2a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:40.954841 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab96fb75e716433f857cb9a4c0d7fb2a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:40.961663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab96fb75e716433f857cb9a4c0d7fb2a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.028619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab96fb75e716433f857cb9a4c0d7fb2a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.040702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab96fb75e716433f857cb9a4c0d7fb2a'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.047467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1370/2764] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:54:40 2022] DELETE /identity/v3/users/ab96fb75e716433f857cb9a4c0d7fb2a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:41.314107 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:41.324500 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71ea8-6447-4793-b0be-53810c79d36e service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:41.384292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:41.385157 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:41.385443 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:41.385721 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] PATH_INFO: `/v3/projects/6329f898314d44d2b6c264d8224a8c46` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:41.386956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6329f898314d44d2b6c264d8224a8c46)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:41.388854 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:41.515008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6329f898314d44d2b6c264d8224a8c46', 'request_id': 'req-56368efc-8509-4b66-add0-2a3fda5ef645'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.521214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6329f898314d44d2b6c264d8224a8c46', 'request_id': 'req-56368efc-8509-4b66-add0-2a3fda5ef645'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.593650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6329f898314d44d2b6c264d8224a8c46', 'request_id': 'req-56368efc-8509-4b66-add0-2a3fda5ef645'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.605125 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] The token cache is being invalidate because project 6329f898314d44d2b6c264d8224a8c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:54:41.605418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.606078 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.607660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1395/2765] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:54:41 2022] DELETE /identity/v3/projects/6329f898314d44d2b6c264d8224a8c46 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:41.611918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:41.613332 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:41.626347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a840ea-dbed-436a-be65-83d6fe2e9ac3 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:41.681573 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:41.682448 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:41.682744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:41.683061 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] PATH_INFO: `/v3/users/b58ebc02f5224524a617f13b2faed414` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:41.688634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58ebc02f5224524a617f13b2faed414)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:41.688634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:41.869786 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58ebc02f5224524a617f13b2faed414'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.870473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58ebc02f5224524a617f13b2faed414'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.872159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:41.878352 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58ebc02f5224524a617f13b2faed414'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.959567 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58ebc02f5224524a617f13b2faed414'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.972043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-cf7b312a-ee69-4a36-856b-259b6cc80e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58ebc02f5224524a617f13b2faed414'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:41.977920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1371/2766] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:54:41 2022] DELETE /identity/v3/users/b58ebc02f5224524a617f13b2faed414 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:42.236670 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:42.238110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:42.251808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56368efc-8509-4b66-add0-2a3fda5ef645 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:42.265231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:42.312382 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:42.313261 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.313539 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.313813 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] PATH_INFO: `/v3/projects/e606c1202c254c32ba913a2a49cf30ca` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.315030 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e606c1202c254c32ba913a2a49cf30ca)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:42.317325 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:42.422404 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e606c1202c254c32ba913a2a49cf30ca', 'request_id': 'req-4a043b24-4c80-4763-bdf1-4b73714fc050'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:42.428731 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e606c1202c254c32ba913a2a49cf30ca', 'request_id': 'req-4a043b24-4c80-4763-bdf1-4b73714fc050'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:42.502521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e606c1202c254c32ba913a2a49cf30ca', 'request_id': 'req-4a043b24-4c80-4763-bdf1-4b73714fc050'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:42.514177 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] The token cache is being invalidate because project e606c1202c254c32ba913a2a49cf30ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:54:42.514466 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:42.515126 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-4a043b24-4c80-4763-bdf1-4b73714fc050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:54:42.518304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1396/2767] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:54:42 2022] DELETE /identity/v3/projects/e606c1202c254c32ba913a2a49cf30ca => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:42.523221 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.526681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.526681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.543110 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:54:42.544987 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:42.607880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1372/2768] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:54:42.669329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.669627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.669936 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.688724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:54:42.689984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:42.698342 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1397/2769] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:54:42.702538 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:42.712160 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a830aa-ac70-4d77-b645-dc85ecbf88d3 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:42.717484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:42.718181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.718467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.718742 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.719519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:54:42.719859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:42.721564 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:42.734161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1373/2770] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:54:42.738214 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:42.747414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700b678-0f42-438e-ac94-0d87da6db10f None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:42.752088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:42.752741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.753024 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.753300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.753979 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:42.755891 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:42.868873 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1398/2771] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:54:42 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:54:42.871827 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:42.881747 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3de-c6c5-4d2e-bf0e-f12d98050cc9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:42.941490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:42.942074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:42.942362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:42.942643 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:42.943386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:42.945301 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:43.048362 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1374/2772] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:54:42 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:54:43.050027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:43.060800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e46da-5600-4f48-a4aa-092188054658 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:43.067150 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:43.067759 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:43.068044 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:43.068324 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:43.069062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:43.070665 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:43.082378 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1399/2773] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:54:43.086444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:43.097557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62cf8d-520b-4b1e-af8d-4ed0956bd5ca admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:43.102890 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:43.103601 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:43.104042 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:43.104333 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:43.105295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:43.106947 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c6cca-f54c-4657-ace4-53a7610189c9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:43.117828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1375/2774] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:54:43.123267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:43.133196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c396449-f615-490b-a2f0-7062c74af026 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:43.138299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:43.139212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:43.139497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:43.139799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] PATH_INFO: `/v3/projects/6e652ff475764534b6cc03cb64b7d069/users/19fc9ffd436d4068a89686da8b4f2997/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:43.155423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e652ff475764534b6cc03cb64b7d069, user_id=19fc9ffd436d4068a89686da8b4f2997, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:43.158110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:43.218004 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1400/2775] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:54:43 2022] PUT /identity/v3/projects/6e652ff475764534b6cc03cb64b7d069/users/19fc9ffd436d4068a89686da8b4f2997/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:43.223372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef960-ef75-4786-901f-c8bc97adb650 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:43.223787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef960-ef75-4786-901f-c8bc97adb650 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:43.224174 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef960-ef75-4786-901f-c8bc97adb650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:43.242192 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-718ef960-ef75-4786-901f-c8bc97adb650 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:54:43.243555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-718ef960-ef75-4786-901f-c8bc97adb650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:43.271116 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:43.310971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1376/2776] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:54:43.318434 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:43.320331 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:43.366867 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:43.383218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290abf-0af0-494e-bab0-0ea5aff2e673 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:43.387135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:43.388375 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:43.388675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:43.388956 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:43.399295 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:43.404347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:43.412386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1401/2777] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:54:49.299240 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.299765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.299900 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.332799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] MFA Rules not processed for user `19fc9ffd436d4068a89686da8b4f2997`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:54:49.334053 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:49.407215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1377/2778] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:54:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:54:49.411671 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.424820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c0d10-bc9b-45fd-97f2-1bdb661d8c43 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:49.431258 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:49.436838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.436838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.436838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.436838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:49.437320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:49.571503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1402/2779] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:54:49 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:54:49.575973 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.587768 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d04b50-d1a0-4c1a-83fc-5e2017e8bdfc None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:49.648670 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:49.649354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.649637 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.649918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.650780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:49.652711 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:49.743534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1378/2780] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:54:49 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:54:49.747398 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.757641 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafae25-b137-4a0d-a0b9-c54d07f39de6 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:49.762320 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:49.763516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.763806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.764093 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.764824 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:49.766426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:49.777329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1403/2781] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:54:49.782068 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.792006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daec08a-ff6b-40dd-aaa5-0c24ae6d1f61 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:49.796610 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:49.797482 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.797764 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.798037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] PATH_INFO: `/v3/projects/62faff752589452fa5054aa6516f0889/users/cf9c8be05b0e4b148342ec721d4a238e/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.812710 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=62faff752589452fa5054aa6516f0889, user_id=cf9c8be05b0e4b148342ec721d4a238e, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:49.815380 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:49.901211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1379/2782] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:54:49 2022] PUT /identity/v3/projects/62faff752589452fa5054aa6516f0889/users/cf9c8be05b0e4b148342ec721d4a238e/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:54:49.905611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.915643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ccc16-fab9-4413-b2cc-a76fefc418af admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:49.972041 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:49.972708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:49.972996 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:49.973276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:49.973998 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:49.975634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe14a-0bd1-4ee5-9e7b-41c27de021f8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:49.985251 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1404/2783] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:54:49.989391 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:49.999531 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b572d4-d4a4-430d-8316-547dd200e9ab admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:50.004100 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:50.004971 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:50.005255 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:50.005532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] PATH_INFO: `/v3/projects/62faff752589452fa5054aa6516f0889/users/cf9c8be05b0e4b148342ec721d4a238e/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:50.007509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62faff752589452fa5054aa6516f0889, user_id=cf9c8be05b0e4b148342ec721d4a238e, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:50.010175 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:50.021708 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1380/2784] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:54:49 2022] PUT /identity/v3/projects/62faff752589452fa5054aa6516f0889/users/cf9c8be05b0e4b148342ec721d4a238e/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:54:56.240634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e330-b7f7-4280-b145-feb2402a890b None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:56.241112 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e330-b7f7-4280-b145-feb2402a890b None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:56.241209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e330-b7f7-4280-b145-feb2402a890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:56.280554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b570e330-b7f7-4280-b145-feb2402a890b None None] MFA Rules not processed for user `cf9c8be05b0e4b148342ec721d4a238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:54:56.280554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b570e330-b7f7-4280-b145-feb2402a890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:56.346808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1405/2785] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:54:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:54:56.362816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:54:56.364634 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:54:56.379200 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:54:56.438067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2c1b1-918c-45ef-ae1f-b5d4d6a80615 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:54:56.443902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:54:56.444552 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:54:56.444842 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:54:56.445123 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:54:56.500285 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:54:56.501895 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:54:56.510762 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1381/2786] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:54:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:55:06.343905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:07.748774 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=50 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:17.884922 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:17.885415 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:17.885542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:17.902905 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] MFA Rules not processed for user `19fc9ffd436d4068a89686da8b4f2997`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:55:17.904629 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:17.911493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1406/2787] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:55:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:55:18.215268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:55:18.268214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b9463-9102-4bc6-9561-668ed75d538f service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:55:18.271767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:55:18.272401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:18.272681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:18.272961 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:18.274750 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:18.284092 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:55:18.285507 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c82d23-cec2-4907-bbc1-25ccc335e7f5 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:55:18.293408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1382/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:55:21.042552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:55:21.043914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:21.114322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3203ec-a955-4329-b7f6-7b4e7d2e64e3 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:55:21.119417 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:55:21.120070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:21.120358 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:21.120642 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:21.122430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:21.137009 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:55:21.140344 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:55:21.149953 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1407/2789] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:55:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:55:28.289928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=26 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:28.643504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:28.643795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:28.644075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:28.664818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] MFA Rules not processed for user `0251939fa06a45da898a12a37ac91163`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:55:28.666387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:28.728196 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1383/2790] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:55:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:55:31.145138 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=35 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:38.720833 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=25 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:39.238643 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:55:39.250786 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057f369-0f47-4872-ac42-a20325e3f6d6 service placement] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:55:39.256477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:55:39.257131 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:39.257424 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:39.257708 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:39.259513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:39.272328 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:55:39.273929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:55:39.285586 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1408/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:55:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:55:42.277802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:55:42.279849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:55:42.292790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efb626-9fe5-41d6-a6be-03c438f42ee5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:55:42.296325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:55:42.296998 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:55:42.297278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:55:42.297550 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:55:42.307144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:55:42.316964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:55:42.317849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1384/2792] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:55:49.280853 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:55:52.313683 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=6 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:56:46.999910 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:56:47.058571 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8622-2d92-4efb-a1a3-d2c66563aa3b service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:56:47.063817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:56:47.064481 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:56:47.064746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:56:47.065027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:56:47.066801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:56:47.102082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:56:47.103866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:56:47.113958 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1409/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:56:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:56:57.109527 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=36 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:02.227922 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:02.269851 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8fdaa-6bce-4968-a689-d0b2f0af9548 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:02.275457 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:02.277296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:02.277296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:02.277296 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:02.278411 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:02.290477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:02.292079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:02.301767 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1385/2794] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:02.341514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:02.343977 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:02.440010 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4c05d-55cd-4a69-b06a-50a0bd2df7d8 service placement] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:02.444941 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:02.445806 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:02.446087 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:02.446365 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] PATH_INFO: `/v3/users/0251939fa06a45da898a12a37ac91163` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:02.447796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0251939fa06a45da898a12a37ac91163)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:02.449695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:02.620060 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0251939fa06a45da898a12a37ac91163'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:02.620722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0251939fa06a45da898a12a37ac91163'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:02.627618 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0251939fa06a45da898a12a37ac91163'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:02.703758 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0251939fa06a45da898a12a37ac91163'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:02.716149 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-998e13b9-13d0-4015-8606-a02bf3a51d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0251939fa06a45da898a12a37ac91163'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:02.722212 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1410/2795] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:02 2022] DELETE /identity/v3/users/0251939fa06a45da898a12a37ac91163 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:02.963614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:02.978234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5f7dd-8948-41d6-9129-bda7317713a2 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.031832 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.032694 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.032981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.033261 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] PATH_INFO: `/v3/projects/7caa2a2f7c644b16a76b9dff92a2b2e8` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.034449 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7caa2a2f7c644b16a76b9dff92a2b2e8)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.036410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.149212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7caa2a2f7c644b16a76b9dff92a2b2e8', 'request_id': 'req-78bd2de6-ad31-40d9-ba5e-320485085cd2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:03.154727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7caa2a2f7c644b16a76b9dff92a2b2e8', 'request_id': 'req-78bd2de6-ad31-40d9-ba5e-320485085cd2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:03.218620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7caa2a2f7c644b16a76b9dff92a2b2e8', 'request_id': 'req-78bd2de6-ad31-40d9-ba5e-320485085cd2'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:03.230788 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] The token cache is being invalidate because project 7caa2a2f7c644b16a76b9dff92a2b2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:03.231426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:03.231779 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-78bd2de6-ad31-40d9-ba5e-320485085cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:03.233387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1386/2796] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:02 2022] DELETE /identity/v3/projects/7caa2a2f7c644b16a76b9dff92a2b2e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:03.238749 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.239065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.239354 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.294502 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:03.295838 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:03.308680 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:03.354236 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1411/2797] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:03.361877 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.362170 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.362477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.381281 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:03.382504 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:03.390284 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1387/2798] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:03.396534 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.409479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835d2a8-7a01-49a5-b182-5be1ef48ac09 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.414801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.415488 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.415775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.416055 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.416778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:57:03.417089 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.418730 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.430269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1412/2799] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:57:03.434439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.443656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9abdde-d1fc-4c1c-9a84-ea1c3e72be73 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.448644 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.449286 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.449569 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.449848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.450539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.452445 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.534965 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1388/2800] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:03.539675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.549834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a6330-212f-4d50-8772-2c6c388601d9 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.606347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.607049 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.607343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.607628 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.608317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.610238 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.699007 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1413/2801] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:03.702718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.713534 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9a1a-d70d-4673-a641-86882a09b5a8 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.718145 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.718826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.719143 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.719423 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.720144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.721738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.732555 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1389/2802] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:03.736513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.746505 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48218f-84b9-4af0-9268-a32b89740e60 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.751239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.751902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.752192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.752480 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.753211 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.754855 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbca7f-fc4a-41e5-b5bf-e452ae34cb52 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.764789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1414/2803] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:03.768966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:03.779358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b58f-caab-4521-9d7a-76d78b1ded9b admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:03.783944 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:03.784818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.785155 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.785386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] PATH_INFO: `/v3/projects/fc59cbb81c7c45a3ace1bb420fd2118b/users/e687760b21b14295b43f59b9dd38fbaa/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.787627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:03.802976 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc59cbb81c7c45a3ace1bb420fd2118b, user_id=e687760b21b14295b43f59b9dd38fbaa, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:03.805652 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:03.887374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1390/2804] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:03 2022] PUT /identity/v3/projects/fc59cbb81c7c45a3ace1bb420fd2118b/users/e687760b21b14295b43f59b9dd38fbaa/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:03.897231 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:03.897553 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:03.899193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:03.924224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] MFA Rules not processed for user `e687760b21b14295b43f59b9dd38fbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:03.926790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:03.988739 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1415/2805] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:03.992894 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.056501 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4822da-4b17-4563-8116-ed48a3cdb8b2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.061190 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.061844 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.062129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.062419 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.063112 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.064996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.147236 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1391/2806] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:03 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:04.152027 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.162856 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fff75d-2117-4e2b-8cc7-8202d992d3df None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.174386 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:04.220414 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.221084 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.221369 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.221648 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.222387 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.224322 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.320433 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1416/2807] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:04 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:04.324593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.335513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ec14d-f0f3-4e6c-b737-5abd9b867fab admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.340295 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.340948 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.341238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.341519 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.342315 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.343934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.355006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1392/2808] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:04.359056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.368973 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75666cb-8f95-46d6-b2cf-7b8c23a03594 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.373513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.374383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.374667 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.374976 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] PATH_INFO: `/v3/projects/77cef4498fd2413ea1e78be1c2b808f7/users/0a32d1e95fed417ea69573bb15817da6/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.389528 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cef4498fd2413ea1e78be1c2b808f7, user_id=0a32d1e95fed417ea69573bb15817da6, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.392209 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.487494 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1417/2809] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:04 2022] PUT /identity/v3/projects/77cef4498fd2413ea1e78be1c2b808f7/users/0a32d1e95fed417ea69573bb15817da6/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:04.491417 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.501468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cefa1-3cbf-4d80-80d1-8402a92468bd admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.558503 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.559198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.559490 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.559772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.560510 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.562098 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e4643-c1f3-4867-a4c5-312eae573346 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.572526 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1393/2810] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:04.576565 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.587767 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c2301-c4b7-4178-8201-3c08aeece818 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.592454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.593337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.593623 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.593907 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] PATH_INFO: `/v3/projects/77cef4498fd2413ea1e78be1c2b808f7/users/0a32d1e95fed417ea69573bb15817da6/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.596062 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cef4498fd2413ea1e78be1c2b808f7, user_id=0a32d1e95fed417ea69573bb15817da6, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.598676 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.608964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1418/2811] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:04 2022] PUT /identity/v3/projects/77cef4498fd2413ea1e78be1c2b808f7/users/0a32d1e95fed417ea69573bb15817da6/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:04.620237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.620529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.620812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.647111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] MFA Rules not processed for user `0a32d1e95fed417ea69573bb15817da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:04.650966 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:04.690642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:04.717949 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1394/2812] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:04.729118 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:04.730810 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:04.814848 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917e6e-4c1d-4bfc-b506-85dc8ccefa74 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:04.818608 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:04.819868 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:04.820154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:04.820432 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:04.830609 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:04.832329 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:04.842851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1419/2813] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:05.026128 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:05.035591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d352f-ed2e-4e0f-948f-097f30566f2d None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:05.039118 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:05.039749 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:05.040032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:05.040308 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:05.097156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:05.098498 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4a3c4-013a-4855-bec9-58a3160ac41d service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:05.108680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1395/2814] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:14.838451 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=72,keystone:INSERT=5 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:15.104408 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=38 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:16.617042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:16.678347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2525-4f93-4369-a0cf-028b312512e9 service cinder] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:16.683732 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:16.684548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:16.684828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:16.685104 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] PATH_INFO: `/v3/users/e687760b21b14295b43f59b9dd38fbaa` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:16.686296 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e687760b21b14295b43f59b9dd38fbaa)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:16.688292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:16.820172 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e687760b21b14295b43f59b9dd38fbaa'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:16.820915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e687760b21b14295b43f59b9dd38fbaa'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:16.828193 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e687760b21b14295b43f59b9dd38fbaa'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:16.893068 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e687760b21b14295b43f59b9dd38fbaa'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:16.906634 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e687760b21b14295b43f59b9dd38fbaa'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:16.913497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1420/2815] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:16 2022] DELETE /identity/v3/users/e687760b21b14295b43f59b9dd38fbaa => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:16.918067 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:16.918681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:16.918681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:16.941476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:16.941476 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:16.995324 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1396/2816] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:17.004893 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:17.006337 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:17.083199 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487d80e-6f76-442c-bf2f-37f384c9cb43 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:17.087400 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:17.088065 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:17.088356 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:17.088640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:17.097666 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:17.099123 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:17.108974 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1421/2817] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:17.482565 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:17.484111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:17.496013 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb15a4d-206b-4090-bf38-4026bcae9987 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:17.500656 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:17.501514 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:17.501799 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:17.502078 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] PATH_INFO: `/v3/projects/fc59cbb81c7c45a3ace1bb420fd2118b` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:17.503300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc59cbb81c7c45a3ace1bb420fd2118b)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:17.505215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:17.607120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc59cbb81c7c45a3ace1bb420fd2118b', 'request_id': 'req-1f853e9b-8e5e-4c9b-9611-5eeedb360079'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.612611 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc59cbb81c7c45a3ace1bb420fd2118b', 'request_id': 'req-1f853e9b-8e5e-4c9b-9611-5eeedb360079'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.681079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc59cbb81c7c45a3ace1bb420fd2118b', 'request_id': 'req-1f853e9b-8e5e-4c9b-9611-5eeedb360079'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.692161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] The token cache is being invalidate because project fc59cbb81c7c45a3ace1bb420fd2118b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:17.692451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.693262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.695525 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1397/2818] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:17 2022] DELETE /identity/v3/projects/fc59cbb81c7c45a3ace1bb420fd2118b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:17.699960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:17.704792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:17.710064 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd5c7-a3fe-44dd-bfc9-edf6f4f82349 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:17.765736 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:17.766548 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:17.766834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:17.767147 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] PATH_INFO: `/v3/users/0a32d1e95fed417ea69573bb15817da6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:17.768370 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a32d1e95fed417ea69573bb15817da6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:17.770281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:17.911514 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a32d1e95fed417ea69573bb15817da6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.912168 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a32d1e95fed417ea69573bb15817da6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.919052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a32d1e95fed417ea69573bb15817da6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.972385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a32d1e95fed417ea69573bb15817da6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.984446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-9531c1cf-2351-4f13-9052-de2d282db9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a32d1e95fed417ea69573bb15817da6'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:17.990299 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1422/2819] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:17 2022] DELETE /identity/v3/users/0a32d1e95fed417ea69573bb15817da6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:18.368986 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:18.370596 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:18.383620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853e9b-8e5e-4c9b-9611-5eeedb360079 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:18.441048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:18.441906 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:18.442194 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:18.442472 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] PATH_INFO: `/v3/projects/77cef4498fd2413ea1e78be1c2b808f7` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:18.443669 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77cef4498fd2413ea1e78be1c2b808f7)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:18.445615 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:18.534360 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:18.550730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77cef4498fd2413ea1e78be1c2b808f7', 'request_id': 'req-23a9a11b-b995-445c-b66e-2a708f33210e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:18.556300 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77cef4498fd2413ea1e78be1c2b808f7', 'request_id': 'req-23a9a11b-b995-445c-b66e-2a708f33210e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:18.647703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77cef4498fd2413ea1e78be1c2b808f7', 'request_id': 'req-23a9a11b-b995-445c-b66e-2a708f33210e'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:18.658981 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] The token cache is being invalidate because project 77cef4498fd2413ea1e78be1c2b808f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:18.659287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:18.659918 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-23a9a11b-b995-445c-b66e-2a708f33210e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:18.661572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1398/2820] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:18 2022] DELETE /identity/v3/projects/77cef4498fd2413ea1e78be1c2b808f7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:18.667196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:18.667490 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:18.673042 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:18.685332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:18.686552 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:18.747611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1423/2821] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:57:18.755682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:18.756002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:18.756289 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:18.775473 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:18.776740 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:18.784728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1399/2822] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:18.789445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:18.799144 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1af9-dc80-46e4-97fd-5c3eba1f9387 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:18.803818 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:18.804484 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:18.804775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:18.805059 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:18.805789 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80308) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:57:18.806103 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:18.807741 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:18.819899 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1424/2823] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:57:18.824302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:18.833724 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60621f6-999a-4b37-ba93-9012b4ff3f42 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:18.838486 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:18.839199 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:18.839493 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:18.839775 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:18.840500 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:18.842400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:18.929139 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1400/2824] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:18.933572 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:18.943691 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350dbe1-6842-4187-902b-7c8b9c7583fa admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:18.948567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:19.001627 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.002332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.002620 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.002902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.003649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.005630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.070409 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1425/2825] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:18 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:19.074469 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.085386 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa44e1-327c-4b54-ab28-8793810a30b9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.090006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.090657 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.090967 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.091249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.091975 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.093561 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.104230 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1401/2826] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:19.108034 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.119513 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655f28d-d466-4602-b8b4-8d089e5cb708 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.124101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.124766 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.125056 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.125332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.126063 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.127719 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c159d-9489-4650-bfc9-6fa02c949b06 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.137722 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1426/2827] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:19.141863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.152242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990f03-8a8a-4b02-9c5c-5b33b000145e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.156881 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.157753 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.158043 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.158328 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] PATH_INFO: `/v3/projects/c38179c20d6a480ab96469cbd8200498/users/d546e95b6ef84e16bf995b978a003654/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.173855 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38179c20d6a480ab96469cbd8200498, user_id=d546e95b6ef84e16bf995b978a003654, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.176551 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.264620 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1402/2828] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:19 2022] PUT /identity/v3/projects/c38179c20d6a480ab96469cbd8200498/users/d546e95b6ef84e16bf995b978a003654/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:19.281305 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.281602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.281883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.308430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] MFA Rules not processed for user `d546e95b6ef84e16bf995b978a003654`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:19.309632 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:19.373128 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1427/2829] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:19.377738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.440120 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280168-0dce-4bef-9eda-f809532cba68 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.445406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.446039 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.446326 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.446602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.447321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.449591 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.516509 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1403/2830] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:19 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:57:19.520649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.531792 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f969c-59aa-4313-9742-69fa6762d87e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.588314 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.588988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.589276 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.589559 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] PATH_INFO: `/v3/users` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.590250 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.592192 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.687454 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1428/2831] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:19 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:19.691602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.702142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c91e-be7e-42c3-8b7c-362e118cc9d6 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.706790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.707470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.707757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.708032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.708752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.710340 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.720599 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1404/2832] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:19.724778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.735269 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540546c-eea1-48cf-87ff-746d3ec61924 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.739846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.740725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.741674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.742297 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] PATH_INFO: `/v3/projects/7053b2da4f7f42c6979c4984383fd6da/users/28134d1dd4da4f748a30acd0c9761708/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.744566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:19.757908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7053b2da4f7f42c6979c4984383fd6da, user_id=28134d1dd4da4f748a30acd0c9761708, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.760624 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.824303 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1429/2833] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:19 2022] PUT /identity/v3/projects/7053b2da4f7f42c6979c4984383fd6da/users/28134d1dd4da4f748a30acd0c9761708/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:19.828389 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.838222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948698-01c0-439a-bddb-5970102bb9f9 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.857816 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:19.894934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.895614 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.895902 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.896182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.896913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.899096 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7689b4-95d9-4150-a5b5-9862e4ac282c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.909158 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1405/2834] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:19.914082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:19.925013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953b508-3107-41ae-8cbd-60d1a552cf9c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:19.929392 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:19.930274 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.930561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.930839 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] PATH_INFO: `/v3/projects/7053b2da4f7f42c6979c4984383fd6da/users/28134d1dd4da4f748a30acd0c9761708/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.932908 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7053b2da4f7f42c6979c4984383fd6da, user_id=28134d1dd4da4f748a30acd0c9761708, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:19.936469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:19.951800 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1430/2835] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:19 2022] PUT /identity/v3/projects/7053b2da4f7f42c6979c4984383fd6da/users/28134d1dd4da4f748a30acd0c9761708/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:19.966676 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:19.966988 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:19.967268 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:19.993485 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] MFA Rules not processed for user `28134d1dd4da4f748a30acd0c9761708`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:19.994684 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:20.059015 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1406/2836] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:20.071923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:20.073892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:20.129927 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc9ef9-02cd-41cc-b3cf-1339f637f26d admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:20.134321 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:20.134987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:20.135278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:20.135556 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:20.144582 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:20.146006 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6a8f1-f942-4875-a6c6-5386072da29e service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:20.155846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1431/2837] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:20.183953 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:20.245079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916ad5-06fc-4834-94fc-c047849134a9 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:20.249736 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:20.250605 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:20.250892 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:20.251198 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] PATH_INFO: `/v3/users/d546e95b6ef84e16bf995b978a003654` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:20.252410 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d546e95b6ef84e16bf995b978a003654)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:20.254330 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:20.443259 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd546e95b6ef84e16bf995b978a003654'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:20.443913 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd546e95b6ef84e16bf995b978a003654'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:20.445172 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:20.452004 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd546e95b6ef84e16bf995b978a003654'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:20.550152 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd546e95b6ef84e16bf995b978a003654'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:20.563713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd546e95b6ef84e16bf995b978a003654'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:20.570374 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1407/2838] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:20 2022] DELETE /identity/v3/users/d546e95b6ef84e16bf995b978a003654 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:20.575611 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:20.575906 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:20.576187 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:20.595119 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:20.596332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:20.659471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1432/2839] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:20.668914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:20.670314 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:20.728180 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914d65-1ebd-40a0-b0a8-e37e61d0a307 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:20.731853 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:20.732521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:20.732805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:20.733082 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:20.741958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:20.743958 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:20.753302 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1408/2840] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:21.110914 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:21.112376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:21.128697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a531d-c918-4425-a787-e1a8f3d8133e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:21.135292 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:21.136205 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:21.136532 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:21.136756 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] PATH_INFO: `/v3/projects/c38179c20d6a480ab96469cbd8200498` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:21.137984 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38179c20d6a480ab96469cbd8200498)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:21.139896 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:21.248639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38179c20d6a480ab96469cbd8200498', 'request_id': 'req-cec26955-6f99-4034-b803-1bc5dbec398f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.254074 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38179c20d6a480ab96469cbd8200498', 'request_id': 'req-cec26955-6f99-4034-b803-1bc5dbec398f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.303794 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38179c20d6a480ab96469cbd8200498', 'request_id': 'req-cec26955-6f99-4034-b803-1bc5dbec398f'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.306336 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:21.315493 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] The token cache is being invalidate because project c38179c20d6a480ab96469cbd8200498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:21.318173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.318173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.318173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1433/2841] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:21 2022] DELETE /identity/v3/projects/c38179c20d6a480ab96469cbd8200498 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:21.321933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:21.331729 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead530e4-55eb-4a7f-891c-632052a944c9 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:21.387025 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:21.387897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:21.388182 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:21.388462 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] PATH_INFO: `/v3/users/28134d1dd4da4f748a30acd0c9761708` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:21.389668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=28134d1dd4da4f748a30acd0c9761708)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:21.391673 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:21.631578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28134d1dd4da4f748a30acd0c9761708'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.632238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28134d1dd4da4f748a30acd0c9761708'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.639075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28134d1dd4da4f748a30acd0c9761708'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.692444 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28134d1dd4da4f748a30acd0c9761708'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.697982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:21.704897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d846ffe0-7664-4e70-af10-17e7db4771cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28134d1dd4da4f748a30acd0c9761708'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:21.710870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1409/2842] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:21 2022] DELETE /identity/v3/users/28134d1dd4da4f748a30acd0c9761708 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:22.099390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.100801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:22.115079 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26955-6f99-4034-b803-1bc5dbec398f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.173471 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.174385 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.174725 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.174988 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] PATH_INFO: `/v3/projects/7053b2da4f7f42c6979c4984383fd6da` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.176167 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7053b2da4f7f42c6979c4984383fd6da)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.178071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.310134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7053b2da4f7f42c6979c4984383fd6da', 'request_id': 'req-90fe7ed5-4873-4dee-a13c-a46495f67ab8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:22.315738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7053b2da4f7f42c6979c4984383fd6da', 'request_id': 'req-90fe7ed5-4873-4dee-a13c-a46495f67ab8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:22.395191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7053b2da4f7f42c6979c4984383fd6da', 'request_id': 'req-90fe7ed5-4873-4dee-a13c-a46495f67ab8'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:22.406390 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] The token cache is being invalidate because project 7053b2da4f7f42c6979c4984383fd6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:22.406675 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:22.407311 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-90fe7ed5-4873-4dee-a13c-a46495f67ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:22.408892 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1434/2843] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:22 2022] DELETE /identity/v3/projects/7053b2da4f7f42c6979c4984383fd6da => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:22.414582 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.414859 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.415177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.433772 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:22.434897 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:22.491977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1410/2844] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:22.500262 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.500580 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.500866 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.520039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:22.521347 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:22.529602 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1435/2845] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:22.533924 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.544151 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3644627-0bd8-4fa9-8b62-06485f8c0c30 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.548735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.549381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.549663 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.549940 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.550752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:57:22.551130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.556755 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.564790 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1411/2846] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:57:22.568993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.578587 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e671bc6-d616-400d-9cfc-e1fd98d6285a None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.583278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.583929 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.584860 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.585310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.586013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.588015 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.644140 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1436/2847] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:22 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:57:22.646532 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.656677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d739-a091-4589-948c-2a6d3522cf37 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.712358 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.712972 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.713257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.713539 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.714293 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.716237 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.801059 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1412/2848] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:22 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:22.805639 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.815985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675778-cbd4-4038-b3e2-6cb3d023cee1 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.820858 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.821523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.821805 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.822082 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.823120 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.824710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.835151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1437/2849] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:22.838810 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.848677 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa8c84-858a-4b87-b8ae-bedc91091ff1 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.853920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.854576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.854863 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.855165 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.855893 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.857467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6313cc-8015-46e9-8d0b-756c15b8dc4a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:22.867251 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1413/2850] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:22.871846 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:22.881430 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db01e1-d27d-48bc-beb8-3d3cd6641a62 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:22.885963 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:22.886829 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:22.887145 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:22.887422 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] PATH_INFO: `/v3/projects/8e5b309841e44ab0880725e27e32728e/users/8ef27d217b434ba0ba76a177dbe47f01/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:22.902038 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5b309841e44ab0880725e27e32728e, user_id=8ef27d217b434ba0ba76a177dbe47f01, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:22.904735 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.015418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1438/2851] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:22 2022] PUT /identity/v3/projects/8e5b309841e44ab0880725e27e32728e/users/8ef27d217b434ba0ba76a177dbe47f01/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:23.027703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.027999 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.028278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.055626 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] MFA Rules not processed for user `8ef27d217b434ba0ba76a177dbe47f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:23.056872 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:23.096624 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:23.119481 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1414/2852] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:23.123975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.149822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:23.187224 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87a158-9055-4afc-97b1-353d3e10c0d5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.194607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.194607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.194607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.194607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.195052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.195790 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.288173 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1439/2853] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:23 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:57:23.292306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.303003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda641d-ed75-433a-a937-4d35f59d3864 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.358870 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.359578 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.359864 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.360144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.360862 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.362802 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.516125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1415/2854] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:23 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:57:23.520061 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.531071 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387f0bc-2868-4f6f-9589-ac9eeeb67792 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.535697 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.536357 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.536650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.536934 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.537659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.539351 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.549778 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1440/2855] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:23.553928 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.563780 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30305e3d-2397-49f0-a8f8-caf905230c26 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.568372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.569243 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.569529 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.569815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] PATH_INFO: `/v3/projects/9ce4865529da44e19f99497681b2af6b/users/364607e63439410e9681262d017fcee8/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.584911 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce4865529da44e19f99497681b2af6b, user_id=364607e63439410e9681262d017fcee8, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.588212 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.675527 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1416/2856] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:23 2022] PUT /identity/v3/projects/9ce4865529da44e19f99497681b2af6b/users/364607e63439410e9681262d017fcee8/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:57:23.679658 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.690029 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16be924-56a3-4b24-aa2c-08d3cd426c8f admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.745784 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.746536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.746826 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.747159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.747885 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.749463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f14ec-a269-4a53-b2be-8127b15106e2 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.759742 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1441/2857] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:23.764006 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.774246 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d090785-6a3e-4abf-a148-2655bbfdf74e admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.778818 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.779765 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.780007 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.780282 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] PATH_INFO: `/v3/projects/9ce4865529da44e19f99497681b2af6b/users/364607e63439410e9681262d017fcee8/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.782277 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce4865529da44e19f99497681b2af6b, user_id=364607e63439410e9681262d017fcee8, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.784933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.794994 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1417/2858] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:23 2022] PUT /identity/v3/projects/9ce4865529da44e19f99497681b2af6b/users/364607e63439410e9681262d017fcee8/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:57:23.804808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.805606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.805887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.832330 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] MFA Rules not processed for user `364607e63439410e9681262d017fcee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:23.833523 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:23.900554 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1442/2859] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:23.911048 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:23.912744 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:23.969023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d55cc0-8717-48e2-b7e8-2cc7a32acf28 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:23.972681 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:23.973318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:23.973602 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:23.973880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:23.982680 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:23.984142 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:23.993916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1418/2860] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:24.088657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:24.098200 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644f7ff-65ad-4ac2-94e3-d279e053a3d4 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:24.102308 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:24.102975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:24.103265 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:24.103541 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:24.116113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:24.160822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:24.162368 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66155c75-7f96-4c4b-820d-792d25eedf8b service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:24.172175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1443/2861] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:27.182846 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:27.221848 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:27.242257 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db2f72-bee6-4fac-97e7-420879b1e0dc service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:27.247556 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:27.248422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:27.248703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:27.248977 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] PATH_INFO: `/v3/users/8ef27d217b434ba0ba76a177dbe47f01` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:27.250141 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef27d217b434ba0ba76a177dbe47f01)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:27.252161 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:27.409581 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef27d217b434ba0ba76a177dbe47f01'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:27.410226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef27d217b434ba0ba76a177dbe47f01'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:27.424395 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef27d217b434ba0ba76a177dbe47f01'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:27.636129 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef27d217b434ba0ba76a177dbe47f01'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:27.648465 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef27d217b434ba0ba76a177dbe47f01'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:27.654821 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1419/2862] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:27 2022] DELETE /identity/v3/users/8ef27d217b434ba0ba76a177dbe47f01 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:27.659423 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:27.664315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:27.664315 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:27.677692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:27.678809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:27.736255 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1444/2863] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:27.745741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:27.747167 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:27.803215 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa72b2-7372-48c8-8fae-fd81565cbd10 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:27.806804 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:27.807466 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:27.807752 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:27.808026 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:27.816730 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:27.818144 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:27.828216 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1420/2864] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:28.278720 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:28.281504 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:28.296662 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb6b7e-70dc-4a92-980b-39a0ea98cee6 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:28.301304 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:28.302163 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:28.302452 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:28.302734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] PATH_INFO: `/v3/projects/8e5b309841e44ab0880725e27e32728e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:28.304154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5b309841e44ab0880725e27e32728e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:28.305920 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:28.424915 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5b309841e44ab0880725e27e32728e', 'request_id': 'req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.431008 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5b309841e44ab0880725e27e32728e', 'request_id': 'req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.503819 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5b309841e44ab0880725e27e32728e', 'request_id': 'req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.515247 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] The token cache is being invalidate because project 8e5b309841e44ab0880725e27e32728e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:28.515536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.516191 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.517734 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1445/2865] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:28 2022] DELETE /identity/v3/projects/8e5b309841e44ab0880725e27e32728e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:28.524508 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:28.531701 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb40f-0ff8-4218-a449-abbfd09e3e0b service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:28.588079 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:28.588989 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:28.589278 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:28.589557 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] PATH_INFO: `/v3/users/364607e63439410e9681262d017fcee8` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:28.590915 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364607e63439410e9681262d017fcee8)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:28.592860 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:28.594422 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:28.838847 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364607e63439410e9681262d017fcee8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.839463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364607e63439410e9681262d017fcee8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.846135 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364607e63439410e9681262d017fcee8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.921354 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364607e63439410e9681262d017fcee8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.933335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-7ef5edfc-357d-45d5-98a6-64e923c64a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364607e63439410e9681262d017fcee8'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:28.939159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1421/2866] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:28 2022] DELETE /identity/v3/users/364607e63439410e9681262d017fcee8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:57:29.334170 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:29.335668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:29.349542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0f59-0cb4-4e64-9c1c-f382ebfda551 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:29.390707 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:29.408942 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:29.409808 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.410098 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.410374 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] PATH_INFO: `/v3/projects/9ce4865529da44e19f99497681b2af6b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.411692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce4865529da44e19f99497681b2af6b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:29.414969 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:29.539537 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce4865529da44e19f99497681b2af6b', 'request_id': 'req-6559b64d-3680-45b5-b281-9ff0685c7b6b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:29.544964 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce4865529da44e19f99497681b2af6b', 'request_id': 'req-6559b64d-3680-45b5-b281-9ff0685c7b6b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:29.636746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce4865529da44e19f99497681b2af6b', 'request_id': 'req-6559b64d-3680-45b5-b281-9ff0685c7b6b'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:29.648110 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] The token cache is being invalidate because project 9ce4865529da44e19f99497681b2af6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:29.650253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:29.650253 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-6559b64d-3680-45b5-b281-9ff0685c7b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:29.650851 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1446/2867] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:29 2022] DELETE /identity/v3/projects/9ce4865529da44e19f99497681b2af6b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:29.655513 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.656002 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.656111 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.673785 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:29.675226 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:29.734074 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1422/2868] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:29.741727 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.742050 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.742332 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.761751 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:29.762986 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:29.770809 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1447/2869] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:29.774916 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:29.784351 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b6cda-d018-4845-a2c9-1f90e1da01a4 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:29.788826 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:29.789470 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.789757 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.790036 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] PATH_INFO: `/v3/domains` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.790763 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80309) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:57:29.791109 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:29.792700 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:29.803936 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1423/2870] 10.222.0.36 () {58 vars in 1152 bytes} [Thu Jun 16 19:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:57:29.808796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:29.817931 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582479b9-f552-44dd-935f-b8391e123a8e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:29.823345 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:29.823991 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.824281 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.824562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.825244 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:29.827190 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:29.912159 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1448/2871] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:29 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:29.916463 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:29.927209 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145390db-f691-4362-a1f6-c79bb147355c admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:29.986735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:29.987440 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:29.987725 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:29.988001 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:29.988702 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:29.990606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.000628 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:30.104484 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1424/2872] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:29 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:57:30.108569 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.119019 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3afd6-ca3b-4545-8d1e-e01ef5e8c466 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.124161 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.124817 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.125101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.125380 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.126113 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.127743 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.138180 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1449/2873] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:30.142070 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.153252 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016e7c8-bea2-4313-9d7a-486a14bae21d admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.156434 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.157084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.157372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.157674 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.158401 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.160058 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4c583-e7f8-4ac0-b0a1-2dc55c4fcb0a admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.171177 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1425/2874] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:30.175343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.186137 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5edae-f59b-4e67-b5c1-be80044c5640 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.190678 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.191592 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.191877 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.192151 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] PATH_INFO: `/v3/projects/be81857db7464187809e3d8efa2b5034/users/3233ef8412314394bb09ec0a0eefddd3/roles/c67c387f18884b4081e6a7754b8319a6` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.207203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be81857db7464187809e3d8efa2b5034, user_id=3233ef8412314394bb09ec0a0eefddd3, role_id=c67c387f18884b4081e6a7754b8319a6)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.209911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.315872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1450/2875] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:30 2022] PUT /identity/v3/projects/be81857db7464187809e3d8efa2b5034/users/3233ef8412314394bb09ec0a0eefddd3/roles/c67c387f18884b4081e6a7754b8319a6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:30.327427 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.331787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.331787 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.356963 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] MFA Rules not processed for user `3233ef8412314394bb09ec0a0eefddd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:30.360184 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:30.428982 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1426/2876] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:30.433436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.495940 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf489-40f8-48c5-933f-19f2c6884110 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.500657 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.503399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.503399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.503399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] PATH_INFO: `/v3/projects` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.503399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.504477 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.514567 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:30.649070 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1451/2877] 10.222.0.36 () {60 vars in 1152 bytes} [Thu Jun 16 19:57:30 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:30.653593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.664176 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf76a9-80a7-4133-b053-5cea9bfac923 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.722604 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.723372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.723593 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.723867 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.724572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.726560 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.869952 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1427/2878] 10.222.0.36 () {60 vars in 1143 bytes} [Thu Jun 16 19:57:30 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:57:30.874039 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.885280 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67074ca-0109-49e6-85e1-5f1721c4c62c admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.889889 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.890544 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.890833 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.891165 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.891883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.893463 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:30.904710 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1452/2879] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:30.908783 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:30.918735 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53925ba-febf-4586-983d-54346824f5b0 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:30.923303 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:30.924181 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:30.924468 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:30.924746 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] PATH_INFO: `/v3/projects/957c4862191842af83714e1af7f3a92b/users/eac7e1a81e934de895d391d0df158ce5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:30.939568 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957c4862191842af83714e1af7f3a92b, user_id=eac7e1a81e934de895d391d0df158ce5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:30.942244 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:31.012713 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1428/2880] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:30 2022] PUT /identity/v3/projects/957c4862191842af83714e1af7f3a92b/users/eac7e1a81e934de895d391d0df158ce5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:31.016689 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:31.027343 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd660be7-b040-4e0e-961c-5e35a937c4b4 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:31.086692 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:31.087399 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:31.087688 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:31.088026 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:31.089267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:31.090898 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b10c63-46a3-4130-96b1-a03c18fbd925 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:31.100872 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1453/2881] 10.222.0.36 () {58 vars in 1121 bytes} [Thu Jun 16 19:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:57:31.105225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:31.115488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036a480-e9a9-49de-aa8c-10558e6cb927 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:31.120619 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:31.121494 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:31.121811 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:31.122085 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] PATH_INFO: `/v3/projects/957c4862191842af83714e1af7f3a92b/users/eac7e1a81e934de895d391d0df158ce5/roles/49095bd2b6a646058d983cc80858b8e3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:31.124084 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957c4862191842af83714e1af7f3a92b, user_id=eac7e1a81e934de895d391d0df158ce5, role_id=49095bd2b6a646058d983cc80858b8e3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:31.126741 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:31.138364 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1429/2882] 10.222.0.36 () {60 vars in 1482 bytes} [Thu Jun 16 19:57:31 2022] PUT /identity/v3/projects/957c4862191842af83714e1af7f3a92b/users/eac7e1a81e934de895d391d0df158ce5/roles/49095bd2b6a646058d983cc80858b8e3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:31.148985 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:31.149277 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:31.149650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:31.177291 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] MFA Rules not processed for user `eac7e1a81e934de895d391d0df158ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:31.178538 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:31.248601 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1454/2883] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:31.260453 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:31.262234 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:31.293032 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:31.320210 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1569-78d1-435d-872d-92b687e8df58 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:31.324805 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:31.325451 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:31.325728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:31.326003 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:31.335606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:31.337156 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:31.346874 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1430/2884] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:31.489040 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:31.499659 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eebe4d-8fe1-4eb2-9dd0-b02cc01c9af2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:31.503456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:31.504101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:31.504383 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:31.504679 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:31.562135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:31.563566 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b9d64-5cc2-4f34-a838-37ba0f458974 service cinder] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:31.568178 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:31.572738 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1455/2885] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:32.260371 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:32.320400 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87841cc-935e-4ccc-acad-19e1df01cd79 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:32.326766 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:32.327653 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:32.327934 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:32.328206 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] PATH_INFO: `/v3/users/3233ef8412314394bb09ec0a0eefddd3` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:32.329407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3233ef8412314394bb09ec0a0eefddd3)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:32.331537 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:32.517914 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3233ef8412314394bb09ec0a0eefddd3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:32.518590 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3233ef8412314394bb09ec0a0eefddd3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:32.525403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3233ef8412314394bb09ec0a0eefddd3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:32.863211 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3233ef8412314394bb09ec0a0eefddd3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:32.876121 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3233ef8412314394bb09ec0a0eefddd3'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:32.881933 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1431/2886] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:32 2022] DELETE /identity/v3/users/3233ef8412314394bb09ec0a0eefddd3 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:32.887203 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:32.887677 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:32.887796 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:32.904698 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] MFA Rules not processed for user `07dde671a0d34c9484dac362c8255554`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:32.905903 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:32.963282 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1456/2887] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:32.972756 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:32.974159 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:33.026347 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:33.029969 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587e51d-d885-4a97-ad1b-44bed21fc4f2 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:33.033688 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:33.034325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:33.034609 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:33.034885 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:33.044225 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:33.045639 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:33.054817 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1432/2888] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:57:33.706526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:33.707975 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:33.720822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfe99b-136f-40a6-8a16-9738f9ea46d2 None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:33.725436 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:33.726310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:33.726593 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:33.726875 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] PATH_INFO: `/v3/projects/be81857db7464187809e3d8efa2b5034` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:33.728135 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be81857db7464187809e3d8efa2b5034)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:33.730206 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:33.886799 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be81857db7464187809e3d8efa2b5034', 'request_id': 'req-7fec8424-13d1-4475-abaf-25929ab2ad63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:33.892310 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be81857db7464187809e3d8efa2b5034', 'request_id': 'req-7fec8424-13d1-4475-abaf-25929ab2ad63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:33.989461 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be81857db7464187809e3d8efa2b5034', 'request_id': 'req-7fec8424-13d1-4475-abaf-25929ab2ad63'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.000646 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] The token cache is being invalidate because project be81857db7464187809e3d8efa2b5034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:34.002501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.002501 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.003122 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1457/2889] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:33 2022] DELETE /identity/v3/projects/be81857db7464187809e3d8efa2b5034 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:34.007195 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:34.017188 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46039b5-63fe-4730-9178-5ece1733dc4e service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:34.074406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:34.075287 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:34.075572 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:34.075849 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] PATH_INFO: `/v3/users/eac7e1a81e934de895d391d0df158ce5` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:34.077041 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac7e1a81e934de895d391d0df158ce5)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:34.078996 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:34.271927 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac7e1a81e934de895d391d0df158ce5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.272515 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac7e1a81e934de895d391d0df158ce5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.280023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac7e1a81e934de895d391d0df158ce5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.349618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac7e1a81e934de895d391d0df158ce5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.363073 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-d7575c9e-e172-4497-bde9-f9ce3854ca68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac7e1a81e934de895d391d0df158ce5'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.369363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1433/2890] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 19:57:34 2022] DELETE /identity/v3/users/eac7e1a81e934de895d391d0df158ce5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:34.780649 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:57:34.782124 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:34.797650 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8424-13d1-4475-abaf-25929ab2ad63 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:57:34.854762 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:57:34.855820 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:34.856300 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:34.856445 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] PATH_INFO: `/v3/projects/957c4862191842af83714e1af7f3a92b` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:34.857561 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957c4862191842af83714e1af7f3a92b)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:57:34.859503 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:57:34.945746 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:34.962154 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957c4862191842af83714e1af7f3a92b', 'request_id': 'req-49a43d10-e4a2-42b2-9dee-454a017a3b56'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:34.967640 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957c4862191842af83714e1af7f3a92b', 'request_id': 'req-49a43d10-e4a2-42b2-9dee-454a017a3b56'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:35.029562 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957c4862191842af83714e1af7f3a92b', 'request_id': 'req-49a43d10-e4a2-42b2-9dee-454a017a3b56'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:35.040542 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] The token cache is being invalidate because project 957c4862191842af83714e1af7f3a92b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80308) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:57:35.040939 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:35.041960 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-49a43d10-e4a2-42b2-9dee-454a017a3b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:57:35.043563 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1458/2891] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 19:57:34 2022] DELETE /identity/v3/projects/957c4862191842af83714e1af7f3a92b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:57:44.365477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:45.104769 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:57:45.623130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] REQUEST_METHOD: `POST` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:57:45.623606 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:57:45.623727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:57:45.640964 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.auth.core [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] MFA Rules not processed for user `19fc9ffd436d4068a89686da8b4f2997`. Rule list: `[]` (Enabled: `True`). {{(pid=80309) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:57:45.642169 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:57:45.697214 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1434/2892] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:57:55.690460 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=25 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:59:46.333497 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] REQUEST_METHOD: `POST` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:46.334376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:46.334376 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:46.355871 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.auth.core [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] MFA Rules not processed for user `19fc9ffd436d4068a89686da8b4f2997`. Rule list: `[]` (Enabled: `True`). {{(pid=80308) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:59:46.357625 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:46.367770 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1459/2893] 10.222.0.36 () {58 vars in 957 bytes} [Thu Jun 16 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:59:46.406107 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:46.407668 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:46.465130 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12ad53-1b11-41d6-8777-bee6977627e5 None None] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:46.468727 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e334de6eee4d5eb8f2fafdb652e9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:46.469363 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:46.469646 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:46.469921 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:46.471695 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:46.558075 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:46.559587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:46.580168 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1435/2894] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:59:46.861052 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:46.862479 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:46.918536 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cfb53-e8f3-4131-adb4-d91721605c1e None None] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:46.922234 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:46.922825 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:46.923139 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:46.923425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:46.925204 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:46.935099 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:46.936526 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:46.944551 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1460/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:59:47.736809 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:47.738406 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:47.749901 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7652c-859c-4647-b77f-a16256460bd0 service cinder] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:47.753521 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:47.754157 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:47.754439 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:47.754720 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:47.756518 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:47.770651 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:47.772307 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:47.784350 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1436/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:59:49.666472 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:49.675218 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379fe5-def2-4994-900f-f19b18d9bafb service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:49.678834 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:49.680724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:49.680724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:49.680724 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:49.682012 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:49.691814 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:49.693273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:49.701570 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1461/2897] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:59:52.734795 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:52.736248 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:52.752655 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64c821-69fd-4bc6-adac-009d05d30f03 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:52.761321 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:52.761917 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:52.762201 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:52.762477 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:52.764381 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:52.778630 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:52.780262 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:52.792627 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1437/2898] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:59:57.069775 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:59:57.083014 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f609-6ebf-45fd-88b4-e8e450275b43 service neutron] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:59:57.088459 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:59:57.089127 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:59:57.089413 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:59:57.089695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:59:57.091465 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:59:57.102013 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:59:57.103607 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:59:57.112431 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1462/2899] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 19:59:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:00:02.787037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=78 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:00:07.108425 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=41 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:01:47.790464 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:01:47.802978 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805203a7-2c47-42f8-bcd8-f876912b288e service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:01:47.808425 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a82e215bee54480b8e4f69ca6352a865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:01:47.809471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:01:47.809471 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:01:47.809773 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:01:47.811728 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:01:47.825492 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:01:47.827136 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:01:47.839737 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1438/2900] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:01:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:01:51.254278 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:01:51.255673 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:01:51.324219 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84031-aad2-486e-8ffd-e4f4f8fc0459 service nova] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:01:51.329695 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:01:51.330822 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:01:51.330989 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:01:51.331169 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:01:51.332911 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:01:51.344588 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:01:51.346136 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:01:51.355878 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1463/2901] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:01:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:01:57.834687 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=6 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:02:01.351660 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=32 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:02:04.230686 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:04.240671 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350d970-23e7-4599-b1f1-41ac6597796f service nova] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:04.244403 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8b8605da7d49cfbdfb0162cbbf2ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:04.245034 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:04.245318 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:04.245592 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:04.247376 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:04.310197 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:04.311682 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:04.321325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1439/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:02:07.819033 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:07.820489 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:07.833919 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e6a60-1905-48d7-9d21-a75cd1724c1e service placement] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:07.839152 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:07.839803 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:07.840158 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:07.840446 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:07.842196 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:07.854181 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:07.855801 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:07.868913 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1464/2903] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:02:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:02:11.397828 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:11.409249 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262a24f-a610-4646-bd79-64425bd1c035 service neutron] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:11.414325 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:11.415046 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] REQUEST_METHOD: `GET` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:11.415335 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:11.415618 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:11.417387 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:11.430242 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:11.432063 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:11.443433 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1440/2904] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:02:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:02:11.519674 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:11.523763 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:11.534995 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67860-3c9f-4582-a6c5-b4b7fc0f744e service placement] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:11.539836 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:11.540705 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:11.540993 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:11.541273 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] PATH_INFO: `/v3/users/19fc9ffd436d4068a89686da8b4f2997` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:11.542456 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fc9ffd436d4068a89686da8b4f2997)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:11.544473 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:11.724175 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fc9ffd436d4068a89686da8b4f2997'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:11.724883 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fc9ffd436d4068a89686da8b4f2997'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:11.731845 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fc9ffd436d4068a89686da8b4f2997'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:11.799669 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fc9ffd436d4068a89686da8b4f2997'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:11.811656 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fc9ffd436d4068a89686da8b4f2997'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:11.817622 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1465/2905] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 20:02:11 2022] DELETE /identity/v3/users/19fc9ffd436d4068a89686da8b4f2997 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 20:02:12.071718 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:12.083238 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711b66e-289f-473a-be5d-81e0698d8146 service placement] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:12.140353 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:12.141213 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:12.142023 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:12.142306 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] PATH_INFO: `/v3/projects/6e652ff475764534b6cc03cb64b7d069` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:12.143576 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e652ff475764534b6cc03cb64b7d069)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:12.145517 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:12.247426 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e652ff475764534b6cc03cb64b7d069', 'request_id': 'req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.253703 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e652ff475764534b6cc03cb64b7d069', 'request_id': 'req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.333738 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e652ff475764534b6cc03cb64b7d069', 'request_id': 'req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.345812 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] The token cache is being invalidate because project 6e652ff475764534b6cc03cb64b7d069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 20:02:12.346179 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.346815 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.348372 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1441/2906] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 20:02:12 2022] DELETE /identity/v3/projects/6e652ff475764534b6cc03cb64b7d069 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 20:02:12.352469 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:12.353923 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:12.367326 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b877-01f5-4716-bc41-22144f0a13e5 admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:12.427101 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:12.428134 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:12.428516 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:12.428902 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] PATH_INFO: `/v3/users/cf9c8be05b0e4b148342ec721d4a238e` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:12.430267 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9c8be05b0e4b148342ec721d4a238e)` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:12.432663 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:12.610088 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9c8be05b0e4b148342ec721d4a238e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.611668 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9c8be05b0e4b148342ec721d4a238e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.619239 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9c8be05b0e4b148342ec721d4a238e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.622837 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:02:12.681557 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9c8be05b0e4b148342ec721d4a238e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.693521 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.notifications [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9c8be05b0e4b148342ec721d4a238e'} {{(pid=80308) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:12.699418 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1466/2907] 10.222.0.36 () {58 vars in 1223 bytes} [Thu Jun 16 20:02:12 2022] DELETE /identity/v3/users/cf9c8be05b0e4b148342ec721d4a238e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 20:02:12.960887 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Authenticating user token {{(pid=80309) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:02:12.962384 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.fernet_utils [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80309) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:02:12.975467 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d491e-0cda-4cc0-a56a-b0919b1f9e41 admin admin] Validating token access rules against request {{(pid=80309) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:02:12.983880 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:02:13.033587 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07dde671a0d34c9484dac362c8255554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adf859e3c6024ed7b109d2f0ca77139a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80309) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:02:13.034443 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:02:13.034731 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:02:13.035037 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] PATH_INFO: `/v3/projects/62faff752589452fa5054aa6516f0889` {{(pid=80309) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:02:13.036222 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62faff752589452fa5054aa6516f0889)` {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:02:13.038131 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] RBAC: Authorization granted {{(pid=80309) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:02:13.131418 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62faff752589452fa5054aa6516f0889', 'request_id': 'req-68c99417-b727-4768-bbe0-e70af7c8ad96'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:13.136920 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62faff752589452fa5054aa6516f0889', 'request_id': 'req-68c99417-b727-4768-bbe0-e70af7c8ad96'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:13.227407 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62faff752589452fa5054aa6516f0889', 'request_id': 'req-68c99417-b727-4768-bbe0-e70af7c8ad96'} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:13.241204 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] The token cache is being invalidate because project 62faff752589452fa5054aa6516f0889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80309) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 20:02:13.241488 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:13.242125 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG keystone.notifications [None req-68c99417-b727-4768-bbe0-e70af7c8ad96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80309) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 20:02:13.243672 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: [pid: 80309|app: 0|req: 1442/2908] 10.222.0.36 () {58 vars in 1232 bytes} [Thu Jun 16 20:02:12 2022] DELETE /identity/v3/projects/62faff752589452fa5054aa6516f0889 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 20:02:22.778987 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:02:23.253642 dv-cinder-846173-1-iscsi devstack@keystone.service[80309]: DEBUG dbcounter [-] [80309] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80309) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 20:07:02.948447 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Authenticating user token {{(pid=80308) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 20:07:02.949876 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:07:03.088823 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1ac6c-a6ae-4e74-8d38-54c91f87be8b admin admin] Validating token access rules against request {{(pid=80308) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 20:07:03.146630 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a46335d4b248048b82d04e3665fe55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0871998ca834c5da02ec3173a0e7e37', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80308) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 20:07:03.147317 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] REQUEST_METHOD: `GET` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 20:07:03.147606 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 20:07:03.147887 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80308) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 20:07:03.149712 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.fernet_utils [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80308) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 20:07:03.160828 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 20:07:03.162426 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9ea78-1a52-461d-bbcb-b71fecbd8eb9 service placement] RBAC: Authorization granted {{(pid=80308) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 20:07:03.172714 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: [pid: 80308|app: 0|req: 1467/2909] 10.222.0.36 () {60 vars in 1365 bytes} [Thu Jun 16 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 20:07:13.168232 dv-cinder-846173-1-iscsi devstack@keystone.service[80308]: DEBUG dbcounter [-] [80308] Writing DB stats keystone:SELECT=89 {{(pid=80308) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}